Validation of a web-based ESL test

Abstract

The web offers new opportunities to realize some of the current ideals for interactive language assessment by providing learners information about their language ability at their convenience. If such tests are to be trusted to provide learners with information that might help to improve their language ability, the tests need to undergo validation processes, but validation theory does not offer specific guidance about what should be included in a validity argument. Conventional wisdom suggests that low-stakes tests require less rigorous validation than high-stakes tests, but what are the factors that affect decisions about the validation process for either? Attempting to make these contributing factors explicit, this article examines the ways in which the purpose of a low-stakes web-based ESL (English as a second language) test guided its design and the validation process. The validity argument resulting from the first phase of the validation process is illustrated.

Get full access to this article

View all access options for this article.

References

Alderson, J. C. and Hamp-Lyons, L. 1996: TOEFL preparation courses: a study of washback . Language Testing 13, 280-297 .

Alderson, J. C. and Wall, D. 1993: Does washback exist? Applied Linguistics 14, 115-129 .

Anderson, R. and Davison, A. 1988: Conceptual and empirical bases of readability formulas. In Davison, A. and Green, G. , editors, Linguistic complexity and text comprehension. Hillsdale, NJ: Lawrence Erlbaum , 23-53.

Bachman, L. F. 1990: Fundamental considerations in language testing. Oxford: Oxford University Press .

Bachman, L. F. and Palmer, A. S. 1996: Language testing in practice. Oxford: Oxford University Press .

Biber, D. 1988: Variation across speech and writing. Cambridge: Cambridge University Press .

Biber, D. , Johansson, S. , Leech, G. , Conrad, S. and Finegan, E. 1999: The Longman grammar of spoken and written English. Harlow, Essex: Pearson Education .

Bruce, B. and Rubin, A. 1988: Readability formulas: matching tool and task. In Davison, A. and Green, G. , editors, Linguistic complexity and text comprehension. Hillsdale, NJ: Lawrence Erlbaum , 5-22.

Chall, J. and Dale, E. 1995: Readability revisited. Cambridge, MA: Brookline Books .

10.

Chapelle, C. 1994: Are C-tests valid measures for L2 vocabulary research? Second Language Research 10, 157-187 .

11.

Chapelle, C. A. 1999: Validation in language assessment . Annual Review of Applied Linguistics 19, 254-272 .

12.

Charrow, V. 1988: Readability v. comprehensibility: a case study in improving a real document. In Davison, A. and Green, G. , editors, Linguistic complexity and text comprehension. Hillsdale, NJ: Lawrence Erlbaum , 85-114.

13.

Cherryholmes, C. 1988: Power and criticism: poststructural investigations in education. New York: Teachers College Press .

14.

Cody, R. and Smith, J. 1997: Applied statistics and the SAS programming language. 4th edition. Upper Saddle River, NJ: Prentice Hall .

15.

Conrad, S. 1996: Investigating academic texts with corpus-based techniques: an example from biology . Linguistics and Education 8, 299-326 .

16.

Crain, S. and Shankweiler, D. 1988: Syntactic complexity and reading acquisition. In Davison, A. and Green, G. , editors, Linguistic complexity and text comprehension. Hillsdale, NJ: Lawrence Erlbaum , 167-192.

17.

Cronbach, L. J. 1988: Five perspectives on validation argument. In Wainer, H. and Braun, H. , editors, Test validity. Hillsdale, NJ: Lawrence Erlbaum , 3-17.

18.

Green, G. and Olsen, M. 1988: Conceptual and empirical bases of readability formulas. In Davison, A. and Green, G. , editors, Linguistic complexity and text comprehension. Hillsdale, NJ: Lawrence Erlbaum , 115-140.

19.

Harrison, C. 1999: Readability. In Spolsky, B. , editor, Concise encyclopedia of educational linguistics. Oxford: Elsevier , 428-431.

20.

Horning, A. 1993: The psycholinguistics of readable writing. Norwood, NJ: Ablex .

21.

Linacre, J. 1999: A measurement approach to computer-adaptive testing of reading comprehension. In Chalhoub-Deville, M. , editor, Issues in computer-adaptive testing of reading proficiency. Studies in language testing 10. Cambridge: Cambridge University Press , 182-195.

22.

McNamara, T. 1996: Measuring second language performance. London: Longman .

23.

Macromedia CourseBuilder for Dreamweaver, version 3.0, 1999. [Computer program extension]. www.macromedia.com

24.

Macromedia Dreamweaver, version 3.0, 1999. [Computer program]. www.macromedia.com

25.

Messick, S. 1989: Validity. In Linn, R. L. , editor, Educational measurement. 3rd edition. New York: Macmillan , 13-103.

26.

Pienemann, M. 1998: Language processing and second language development: processability theory. Philadelphia, PA: John Benjamins .

27.

Read, J. 2000: Assessing vocabulary. Cambridge: Cambridge University Press .

28.

Read, J. and Chapelle, C. 2001: A framework for second language vocabulary assessment . Language Testing 18, 1-32 .

29.

Rupp, A. , Garcia, P. and Jamieson, J. 2002: Combining multiple regression and CART to understand difficulty in second language reading and listening comprehension test items . International Journal of Language Testing 1, 185-216 .

30.

Shepard, L. 1993: Evaluating test validity . Review of Research in Education 19, 405-450 .

31.

Ulijn, J. and Strother, J. 1990: The effect of syntactic simplification on reading EST texts as L1 and L2 . Journal of Research in Reading 13, 38-54 .