Operationalizing uncertainty in language testing: an argument in favour of content validity

Abstract

It is argued that five variables associated with language proficiency are all subject to unreliability. They are: the native speaker, the cut-off, the criterion score, the test and the language. The native speaker exists only in an idealized form. The cut-off has at best a local definition and at worst is hugely subjective. The criterion used for validation is itself often less reliable than its predictor. Tests are difficult, perhaps impossible to replicate and therefore to generalise from. Language itself balances the general and the particular (this is the 'language': 'a language' distinction). In all these areas the object of description is elusive and fugitive. Language testing has these problems to an even greater extent because measurement is by its nature prone to error. The elusiveness of precision in language testing is discussed in relation to three major issues of linguistic analysis: universality, combining power and the incorporation of extra language data, and results from a recent validation study of the English Language Testing Service (ELTS) Test are quoted to exemplify that relation ship. It is suggested that the test has attempted overspecification and concluded that language testing should accept the centrality of uncertainty and make it explicit.

Get full access to this article

View all access options for this article.

References

Alderson, J.C. 1979: The cloze procedure and proficiency in English as a foreign language. TESOL Quarterly 13, 219-27.

Alderson, J.C. and Urquhart, A.H. 1983: The effect of student background discipline on comprehension: a pilot study. In Porter, D. and Hughes, A. , editors, Current developments in language testing, London: Academic Press, 121-28.

Criper, C. and Davies, A. 1986: The ELTS Validation Study. Report to the British Council and University of Cambridge Local Examinations Syndicate.

Cronbach, L.J. 1961: Essentials of psychological testing, second edition. London, Harper and Row.

Davies, A. 1983: English language testing in the University of Edinburgh: report on the English Language Battery (ELBA Test) with an analysis of results 1973-70. Unpublished, University of Edinburgh.

Ingram, E. 1973: English standards of foreign students. University of Edinburgh Bulletin.

Munby, J.L. 1978: Communicative syllabus design. Cambridge : Cambridge University Press.

Ross, J.R. 1979: Where's English? In Fillmore, C.J. , Kemper, D. and Wang, W.S-Y. , editors, Individual differences in language ability and language behavior, New York: Academic Press.

Sapir, E. 1921: Language: an Introduction to the study of speech. New York, Harcourt Brace and World.