SLA developmental stages and teachers’ assessment of written French: Exploring Direkt Profil as a diagnostic assessment tool

Abstract

One core area of research in Second Language Acquisition is the identification and definition of developmental stages in different L2s. For L2 French, Bartning and Schlyter (2004) presented a model of six morphosyntactic stages of development in the shape of grammatical profiles. The model formed the basis for the computer program Direkt Profil (Granfeldt et al., 2006), which carries out an automated analysis of the developmental stage of a learner text. The aim of the present study was to explore the relevance of Direkt Profil as a diagnostic assessment tool by comparing Direkt Profil’s automated profile analysis with assessment by trained language teachers.

Data for the present study come from the CEFLE corpus of written L2 French (Ågren, 2008). The learner texts were first analysed for developmental stage by the computer program Direkt Profil. In a second step, seven experienced language teachers of French assessed the same texts. The results indicated relatively high degrees of correlation and showed that the analysis of developmental stage by Direkt Profil could explain 73% of the variance in the teachers’ mean assessments (r² = 0.735). In addition, we concluded that the teachers were in agreement with each other and with the computer program when assessing texts at low and high proficiency levels respectively. The most important variation in the teachers’ assessments was found in texts at intermediate levels, due to an inconsistent use of grammar and vocabulary. One of the advantages of using Direkt Profil as a diagnostic assessment tool is that it provides immediate and detailed feedback indicating how certain types of linguistic structures, correct or incorrect, are related to different stages of development.

Keywords

assessment developmental stages Direkt Profil L2 French language testing linguistic profiling second language acquisition

Get full access to this article

View all access options for this article.

References

Ågren

(2008). A la recherche de la morphologie silencieuse: Sur le dévelopement du pluriel en français L2 écrit. Études Romanes de Lund 84. (Doctoral dissertation). Lund University.

Ågren

Granfeldt

Schlyter

(2012).The growth of complexity and accuracy in L2 French. Past observations and recent applications of developmental stages. In Housen

Kuikken

Vedder

(Eds.), Dimensions of L2 performance and proficiency. Complexity, accuracy and Fluency in SLA (pp. 95–119). Amsterdam/Philadelphia: John Benjamins.

Alderson

J. C.

(2005) Diagnosing foreign language proficiency: the interface between learning and assessment. London: Continuum.

Alderson

J. C.

(2010). Language testing-informed SLA? SLA-informed language testing? EUROSLA Monographs series, 1, 239–248.

Bachman

L. F.

(1988). Language testing–SLA research interfaces. Annual Review of Applied Linguistics, 9, 193–209.

Bachman

L. F.

Cohen

A. D.

(Eds.) (1998). Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press.

Bartning

(2009). Et après le stade 6…? Autour des derniers stades de l’acquisition du français L2. In Bernardini

Egerland

Granfeldt

(Eds.), Mélanges plurilingues offerts à Suzanne Schlyter à l’occasion de son 65^ème anniversaire (pp. 29–49). Lund University: Études Romanes de Lund 85.

Bartning

Martin

Vedder

(Eds.) (2010). Second language acquisition and language testing. EUROSLA Monograph Series, 1.

Bartning

Schlyter

(2004). Stades et itinéraires acquisitionnels des apprenants suédophones en français L2. Journal of French Language Studies, 14, 281–299.

10.

Brindley

(1998). Describing language development? Rating scales and SLA. In Bachman

L. F.

Cohen

A. D.

(Eds.), Interfaces between second language acquisition and language testing research (pp. 112–140). Cambridge: Cambridge University Press.

11.

Clahsen

(1985). Profiling second language development: A procedure for assessing L2 proficiency. In Hyltenstam

Pienemann

(Eds.), Modelling and assessing second language acquisition (pp. 283–331). Clevedon, UK: Multilingual Matters.

12.

Crystal

Fletcher

Garman

(1976). The grammatical analysis of language disability: A procedure for assessment and remediation. Studies in Language Ability and Remediation, 1, London: Edward Arnold.

13.

DiBiase

Kawaguchi

(2002). Exploring the typological plausibility of Processability Theory: Language development in Italian second language and Japanese second language. Second Language Research, 18, 273–302.

14.

Ellis

(2001). Some thoughts on testing grammar: An SLA perspective. In Elder

Brown

Grove

Hill

Iwashita

Lumley

McNamara

O’Loughlin

(Eds.), Experimenting with uncertainty: Essays in honour of Alan Davies (pp. 251–263). Cambridge: Cambridge University Press.

15.

Granfeldt

Nugues

(2007). Evaluating stages of development in second language French: A machine learning approach. In Nivre

Kaalep

H. J.

Muischnek

Koit

(Eds.), NODALIDA 2007 Conference Proceedings (pp. 73–80). Tartu, Estonia.

16.

Granfeldt

Nugues

Ågren

Thulin

Persson

Schlyter

(2006). CEFLE and Direkt Profil: A new computer learner corpus in French L2 and a system for grammatical profiling. In Proceedings of the 5th International Conference on Language Resources and Evaluation (pp. 565–570). Genoa, Italy.

17.

Granfeldt

Nugues

Persson

Kostadinov

Ågren

Schlyter

(2005). Direkt Profil: A system for evaluating texts of second language learners of French based on developmental sequences. In Proceedings of the Second Workshop on Building Educational Applications Using Natural Language Processing, 43rd Annual Meeting of the Association of Computational Linguistics (pp. 53–60). Ann Arbor, MI.

18.

Hawkins

Filipovic

(2012). Criterial features in L2 English: Specifying the reference levels of the Common European Framework. English Profile Studies 1. Cambridge: Cambridge University Press.

19.

Hayes

F. A.

Krippendorff

(2007). Answering the call for a standard reliability measure for coding data. Communication Methods and Measures, 1, 77–89.

20.

Hulstijn

(2007). Fundamental issues in the study of second language acquisition. EUROSLA Yearbook, 7, 191–203.

21.

Hulstijn

(2010). Linking L2 proficiency to L2 acquisition: Opportunities and challenges of profiling research. EUROSLA Monographs series, 1, 233–238.

22.

Hyltenstam

(1985). L2 learners’ variable output and language teaching. In Hyltenstam

Pienemann

(Eds.), Modelling and assessing second language acquisition (pp. 113–136). Clevedon: Multilingual Matters.

23.

Keβler

J.-U.

Liebner

(2011). Diagnosing L2 development. Rapid Profile. In Pienemann

Keβler

J.-U.

(Eds.), Studying Processability Theory (pp. 132–148). Amsterdam/Philadelphia: John Benjamins.

24.

Laufer

Nation

(1995). Vocabulary size and use: Lexical richness in L2 written production. Applied Linguistics, 16(3): 307–322.

25.

Meunier

(1998). Computer tools for the analysis of learner corpora. In Granger

(Ed.), Learner English on computer (pp. 19–37). London/New York: Addison Wesley Longman.

26.

Pallotti

(2010). Doing interlanguage analysis in school contexts. EUROSLA Monographs series, 1, 159–190.

27.

Pienemann

Johnston

Brindley

(1988). Constructing an acquisition-based procedure for second language assessment. Studies in Second Language Acquisition, 10, 217–243.

28.

Pienemann

Mackey

(1993). An empirical study of children’s ESL development and Rapid Profile. In McKay

(Ed.), ESL development. Language and literacy in schools. Vol. 2 (pp. 115–259). Sydney, Australia: Commonwealth of Australia and National Languages and Literacy Institute.

29.

Purpura

J. E.

(2004). Assessing grammar. Cambridge: Cambridge University Press.

30.

Salmoura

Saville

(2010). Exemplifying the CEFR: Criterial features of written learner English from the English Profile Programme. EUROSLA Monographs series, 1, 101–132.

31.

Witten

I. H.

Frank

(2005). Data mining. Practical machine learning tools and techniques. Amsterdam: Elsevier.