Performance consistency in second language acquisition and language testing research: a conceptual gap

Abstract

Arguing from the premise that a language test is a special case of a second language acquisition (SLA) elicitation device, I suggest that SLA and language testing share much common ground in terms of research methods, which have similar properties in that they are both used to make systematic observations of language performances from which inferences can be made about the state of a learner’s interlanguage ability underlying the performance. However, I also argue that whereas the concept of demonstrating validity and reliability has been integrated into how language testing research is conducted, SLA researchers have generally failed to recognize the need to demonstrate these qualities. I compare examples of SLA and language testing research articles in terms of their treatment of validity and reliability and argue:

• that it is important for SLA researchers to provide evidence that the methods they employ to elicit data are appropriate for the purposes intended;

• that the procedures provide stable and consistent data; and, consequently

• that the interpretations they make of the results are justified.

Get full access to this article

View all access options for this article.

References

Alderson, J. C. 1979: The cloze procedure and proficiency in English as a foreign language . TESOL Quarterly 13, 219-227 .

Bachman, L. 1989: Language testing - SLA research interfaces . Annual Review of Applied Linguistics 9, 193-209 .

Bachman, L. 1990: Fundamental considerations in language testing. Cambridge: Cambridge University Press .

Bachman, L. and Cohen, A. 1998: Language testing: SLA interfaces: an update. In Bachman, L. and Cohen, A. , editors, Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press , 1-31.

Bard, E. , Robertson, D. and Sorace, A. 1996: Magnitude estimation of linguistic acceptability . Language 72, 32-68 .

Bley-Vroman, R. and Yoshinaga, N. 2000: The acquisition of multiple wh-questions by high-proficiency non-native speakers of English . Second Language Research 16, 3-26

Carroll, J. B. 1968: The psychology of language testing . In Davies, A. , editor, Language testing symposium: a psycholinguistic perspective. London: Oxford University Press, 46-69 .

Chalhoub-Deville, M. 1995: Deriving oral assessment scales across different tests and rater groups . Language Testing 12, 16-33 .

Chapelle, C. 1998: Construct definition and validity inquiry in SLA research. In Bachman, L. and Cohen, A. , editors, Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press , 32-70.

10.

Douglas, D. 1998: Testing methods in context-based second language research. In Bachman, L. and Cohen, A. , editors, Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press , 141-155.

11.

Douglas, D. 2000: Assessing languages for specific purposes. Cambridge: Cambridge University Press .

12.

Douglas, D. and Selinker, L. 1985: Principles for language tests within the ‘discourse domains’ theory of interlanguage: research, test construction, and interpretation . Language Testing 2, 205-226 .

13.

Douglas, D. and Selinker, L. 1994: Research methodology in context-based second-language research. In Tarone, E. , Gass, S. , and Cohen, A. , editors, Methodologies for eliciting and analyzing language in context. Hillsdale, NJ: Erlbaum , 119-132.

14.

Dube, B. 2000: Where are the minimal trees? Evidence from early Zulu L2 subordination . Second Language Research 16, 233-265 .

15.

Educational Testing Service 2001: Internet web page: http://www.toefl.org/swe-pq.html [8 April].

16.

Elder, C. 1993: The proficiency test for language teachers: Italian, Volume 2: appendices. Melbourne: National Language and Literacy Institute of Australia Language Testing Centre, University of Melbourne .

17.

Gass, S. 1994: The reliability of L2 grammaticality judgements. In Tarone, E. , Gass, S. , and Cohen, A. , editors, Methodologies for eliciting and analyzing language in context. Hillsdale, NJ: Erlbaum , 303-322.

18.

Hoekje, B. and Linnell, K. 1994: ‘Authenticity’ in language testing: evaluating spoken language tests for international teaching assistants . TESOL Quarterly 28, 103-126 .

19.

Hymes, D. 1974: Foundations in sociolinguistics: an ethnographic approach. Philadelphia, PA: University of Pennsylvania Press .

20.

Iowa State University , no date: TEACH Rating Sheet. Xerox. Ames, IA: Iowa State University .

21.

Loevinger, J. 1957: Objective tests as instruments of psychological theory . Psychological Reports 3, 635-694 .

22.

Mandell, P. 1999: On the reliability of grammaticality judgement tests in second language acquisition research . Second Language Research 15: 73-99 .

23.

Messick, S. 1989: Validity. In Linn, R. , editor, Educational measurement (3rd edition). New York: Macmillan , 13-103.

24.

Messick, S. 1995: Validity of psychological assessment: validation of inferences from persons’ responses and performance as scientific inquiry into scoring meaning . American Psychologist 9, 741-749 .

25.

Papp, S. 2000: Stable and developmental optionality in native and non-native Hungarian grammars . Second Language Research 16, 173-200 .

26.

Robertson, D. 2000: Variability in the use of the English article system by Chinese learners of English . Second Language Research 16, 135-172 .

27.

Sasaki, M. 2000: Effects of cultural schemata on students’ test-taking processes for cloze tests: a multiple data source approach . Language Testing 17, 85-114 .

28.

Selinker, L. and Douglas, D. 1989: Research methodology in contextually-based second language research . Second Language Research, 5, 93-126 .

29.

Spelberg, H , de Boer, P. and van den Bos, K. 2000: Item type comparisons of language comprehension tests . Language Testing 17, 311-322 .

30.

Swain, M. 1990: Second language testing and second language acquisition: is there a conflict with traditional psychometrics? In Alatis, J. , editor, Georgetown University Roundtable, 1990. Washington, DC: Georgetown University Press , 401-412.

31.

Tarone, E. 1983: On the variability of interlanguage systems . Applied Linguistics 4, 142-163 .

32.

Tarone, E. 1998: Research on interlanguage variation: implications for language testing. In Bachman, L. and Cohen, A. , editors, Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University Press , 70-89.