On the reliability of grammaticality judgement tests in second language acquisition research

Abstract

The use of grammaticality and metalinguistic judgement tests in second language acquisition (SLA) research has been the subject of considerable scrutiny over the past decade (see, for example, Chaudron, 1983; Birdsong, 1989; Ellis, 1991; Cowan and Hatasa, 1994; Gass, 1994; Davies and Kaplan, 1998).Grammaticality judgement (GJ) test data in research design are used to make inferences about the syntactic structures and rules that constitute learners’ linguistic competence, in the Chomskian sense of the word. One criticism levelled at the use of this type of test, however, is that they are not reliable measures of linguistic competence.

The present study compared GJ test data with dehydrated sentence (DS) test data, an assessment tool commonly used in the L2 (second language) classroom. Data were collected from three levels (second, fourth and sixth semester) of adult L2 learners of Spanish about verb movement (Vmovement). The results from the comparison of the two tests indicated that GJ data are reliable measures of linguistic knowledge.

Get full access to this article

View all access options for this article.

References

Birdsong, D. 1989:Metalinguistic performance and interlanguage competence. New York: Springer .

Bley-Vroman, R. , Felix, S. and Ioup, G. 1988: The accessibility of Universal Grammar in adult language learning .Second Language Research 4, 1–32 .

Chaudron, C. 1983: Research on metalinguistic judgments: A review of theory, methods, and results .Language Learning 33, 343–377 .

Chomsky, N. 1981:Lectures on government and binding. Dordrecht: Foris .

Cowan, R. and Hatasa, Y.A. 1994: Investigating the validity and reliability of native speaker and second-language learner judgments about sentences. In Tarone, E. , Gass, S.M. and Cohen, A. , editors,Research methodology in second language acquisition. Hillsdale, NJ: Lawrence Erlbaum , 287–302.

Davies, W.D. and Kaplan, T.I. 1998: Native speaker vs. L2 learner grammaticality judgements .Applied Linguistics 19, 183–203 .

Downey-Vanover, J. 1994: Adult access to Universal Grammar in second language acquisition. Unpublished doctoral dissertation,University of Maryland, College Park, MD.

Ellis, R. 1991: Grammaticality judgements and second language acquisition .Studies in Second Language Acquisition 13, 161–186 .

Felix, S. 1988: UG-generated knowledge in adult second language acquisition. In Flynn, S. and O’Neil, W. , editors,Linguistic theory in second language acquisition. Dordrecht: Kluwer , 277–294.

10.

Gass, S.M. 1994: The reliability of second-language grammaticality judgments. In Tarone, E. , Gass, S.M. and Cohen, A. , editors,Research methodology in second language acquisition. Hillsdale, NJ: Lawrence Erlbaum , 303–322.

11.

Goss, N. , Ying-Hua, Z. and Lantolf, J.P. 1994: Two heads may be better than one: mental activity in second-language activity judgments. In Tarone, E. , Gass, S.M. and Cohen, A. , editors,Research methodology in second language acquisition. Hillsdale, NJ: Lawrence Erlbaum , 263–286.

12.

Hawkins, R. , Towell, R. and Bazergui, N. 1993: Universal Grammar and the acquisition of French verb movement by native speakers of English .Second Language Research 9, 189–233 .

13.

Hulk, A. 1991: Parameter setting and the acquisition of word order in L2 French .Second Language Research 7, 1–34 .

14.

Johnson, J.S. , Shenkman, K.D. , Newport, E.L. and Medin, D.L. 1996: Indeterminacy in the grammar of adult language learners .Journal of Memory and Language 35, 335–352 .

15.

Leow, R. 1996: Grammaticality judgment tasks and second-language development.Georgetown University Round Table on Languages and Linguistics, 126–139.

16.

Liceras, J. 1993: Los juicios de gramaticalidad en terreno movedizo .Foro Hispànico 6, 15–30 .

17.

Mandell, P.B. 1998: The V-movement parameter and adult L2 learners of Spanish .Spanish Applied Linguistics 2, 169–198 .

18.

Pollock, J.Y. 1989: Verb movement, Universal Grammar, and the structure of IP .Linguistic Inquiry 20, 365–424 .

19.

Schwartz, B. 1993: On explicit and negative data effecting and affecting competence and linguistic behavior .Studies in Second Language Acquisition 15, 147–163 .

20.

Schwartz, B.D. and Sprouse, R.A. 1996: L2 cognitive states and the Full Transfer/Full Access model .Second Language Research 12, 40–72 .

21.

Suñer, M. 1994: V-movement and the licensing of argumental Wh-phrases in Spanish .Natural Language and Linguistic Theory 12, 335–372 .

22.

Torrego, E. 1984: On inversion and some of its effects .Linguistic Inquiry 15, 103–129 .

23.

VanPatten, B. and Mandell, P.B. forthcoming: Variable performance on Grammaticality Judgment Tests: the role of knowledge source. Unpublished manuscript.

24.

White, L. 1985: The pro-drop parameter in adult second language acquisition .Language Learning 35, 47–62 .

25.

White, L. 1988: Island effects in second language acquisition. In Flynn, S. and O’Neil, W. editors,Linguistic theory in second language acquisition. Dordrecht: Kluwer , 144–172.

26.

White, L. 1990a: Another look at the logical problem of foreign language learning: a reply to Bley-Vroman .Linguistic Analysis 20, 50–63 .

27.

White, L. 1990b: The Verb-Movement Parameter in second language acquisition .Language Acquisition 1, 337–360 .

28.

White, L. 1992: Long and short verb movement in second language acquisition . Canadian Journal of Linguistics 37, 273–286 .

29.

Zagona, K. 1988:Verb phrase syntax: a parametric study of English and Spanish. Kluwer: Dordrecht .