This paper uses a discussion of the role of Rasch model IRT in the validation of two sub-tests of the Occupational English Test to argue for the usefulness of IRT as a tool in the exploration of test constructs and also to consider the implications of the empirical analysis presented for the validity of communica tive language tests involving the skills of speaking and writing.
Get full access to this article
View all access options for this article.
References
1.
Alderson, J.C., Candlin, C.N., Clapham, C.M., Martin, D.J., and Weir, C.J.1986: Language proficiency testing for migrant professionals: new directions for the Occupational English Test. A report submitted to the Council on Overseas Professional Qualifications. Lancaster : Institute for English Language Education, University of Lancaster.
2.
Andrich, D.1978: A rating formulation for ordered response categories . Psychometrika43: 561-73.
3.
Bachman, L.F.1990: Fundamental considerations in language testing . Oxford: Oxford University Press .
4.
Brown, J.D. and Bailey, K.M.1984: A categorical instrument for scoring second language writing skills. Language Learning34, 4: 21-42.
5.
Carlson, S.B. and Camp, R.1985: Relationships between direct and indirect measures of writing ability. Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL, March 31-April 4 (ERIC document ED255543).
6.
Clark, J.L.D.1972: Foreign language testing: theory and practice. Philadelphia, PA: Center for Curriculum Development .
7.
Clark, J.L.D. and Clifford, R.T.1988: The FSI/ILR/ACTFL proficiency scales and testing techniques: development, current status and needed research. Studies in Second Language Acquisition10, 2: 129-47.
8.
Daiker, D., Kerek, A. and Morenberg, M.1978: Sentence combining and syntactic maturity in freshman English. College Composition and Communication19, 1:36-41.
9.
Davidson, D.M.1978: Test of ability to subordinate. New York: Language Innovations, Inc.
10.
Davidson, F. and Henning, G.1985: A self-rating scale of English difficulty: Rasch scalar analysis of items and rating categories. Language Testing2: 164-79.
11.
Evola, J., Mamer, E. and Lentz, B.1980: Discrete point versus global scoring for cohesive devices . In Oller, J. W. and Perkins, K. editors, Research in language testing. Rowley, MA: Newbury House , 177-81.
12.
Hamp-Lyons, L.1989: Applying the partial credit model of Rasch analysis: language testing and accountability. Language Testing6: 109-18.
13.
Henning, G.1987: A guide to language testing: development, validation, research. Cambridge, MA: Newbury House.
14.
Henning, G. and Davidson, F.1987: Scalar analysis of composition ratings. In Bailey, K. M., Dale, T. L. and Clifford, R. T., editors, Language testing research. Selected papers from the 1986 colloquium. Monterey, CA: Defense Language Institute, 24-38.
15.
Higgs, T.V. and Clifford, R.1982: The push toward communication. In Higgs, T. V., editor, Curriculum, competence and the foreign language teacher. Skokie, IL: National Textbook Company, 57-79.
16.
Homburg, T.J.1984: Holistic evaluation of ESL compositions: can it be validated objectively?TESOL Quarterly18: 87-107.
Lowe, P.1982: ILR handbook on oral interview testing. Washington, DC: Defense Language Institute / CIA Language School.
19.
McNamara, T.F.1987: Assessing the language proficiency of health professionals. Recommendations for the reform of the Occupational English Test. A report submitted to the Council on Overseas Professional Qualifications. Parkville, Victoria: University of Melbourne, Department of Russian and Language Studies .
20.
— 1988: The development of an English as a Second Language speaking test for health professionals. Part One of a report to the Council on Overseas Professional Qualifications on a consultancy to develop the Occupational English Test. Parkville, Victoria: University of Melbourne, Department of Russian and Language Studies.
21.
— 1989a: The development of an English as a Second Language test of writing skills for health professionals. Part Two of a report to the Council on Overseas Professional Qualifications on a consultancy to develop the Occupational English Test. Parkville, Victoria: University of Melbourne, Department of Russian and Language Studies .
22.
— 1989b: ESP testing: general and particular. In Candlin, C. N. and McNamara, T.F. editors, Language, learning and community. Sydney, NSW: National Centre for English Language Teaching and Research, Macquarie University, 125-42.
23.
— 1990a: The role of Item Response Theory in language test validation . Paper presented at the RELC Regional Seminar on 'Language testing and language programme evaluation', Singapore, April 9-12.
24.
— 1990b: Assessing the second language proficiency of health professionals . PhD thesis, University of Melbourne.
25.
Mullen, K.1977: Using rater judgments in the evaluation of writing proficiency for non-native speakers of English. In Brown, H. D., Yorio, C. A. and Crymes, R. H., editors, Teaching and learning English as a second language: trends in research and practice. Washington, DC: TESOL, 309-20.
26.
Perkins, K.1983: On the use of composition scoring techniques, objective measures, and objective tests to evaluate ESL writing ability. TESOL Quarterly17: 651-71.
27.
Perkins, K. and Konneker, B.H.1981: Testing the effectiveness of sentencecombining in ESL composition. Paper presented at Annual TESOL Convention , Detroit, Michigan.
28.
Pienemann, M. and Johnston, M.1987: Factors influencing the development of language proficiency . In Nunan, D., editor, Applying second language acquisition research. Adelaide, SA: National Curriculum Resource Centre, 45-141.
29.
Pienemann, M., Johnston, M. and Brindley, G.1988: Constructing an acquisition-based procedure for second language assessment. Studies in Second Language Acquisition10: 217-43.
30.
Pollitt, A. and Hutchinson, C.1987: Calibrated graded assessments: Rasch partial credit analysis of performance in writing. Language Testing4: 72-92.
31.
Raffaldini, T.1988: The use of situation tests as measures of communicative ability. Studies in Second Language Acquisition10: 197-216.
32.
Savignon, S.1985: Evaluation of communicative competence: the ACTFL Provisional Proficiency Guidelines. Modern Language Journal69: 129-34.
33.
Shohamy, E.1988: A proposed framework for testing the oral language of second/foreign language learners. Studies in Second Language Acquisition10: 165-79.
34.
Skehan, P.1988: Language testing. Part I. Language Teaching21: 211-21.
35.
— 1989: Language testing. Part II. Language Teaching22: 1-13.
36.
Upshur, J.A. et al. 1966: Michigan test of English language proficiency (Forms A-E). Ann Arbor, Michigan: University of Michigan, Testing and Certification Division, English Language Institute.
37.
Weir, C.1988: Construct validity. In Hughes, A., Porter, D. and Weir, C., editors, EL TS Validation Project: proceedings of a conference held to consider the ELTS Validation Project Report . English Language Testing Service Research Report 1 (ii). London: British Council/ University of Cambridge Local Examinations Syndicate, 15-25.
38.
Wilds, C.P.1975: The oral interview test. In Jones, R. L. and Spolsky, B., editors, Testing language proficiency. Arlington, VA: Center for Applied Linguistics, 29-44.
39.
Wright, B. and Linacre, J.M.1984: Microscale manual for Microscale version 1.2. Westport, CT.: Mediax Interactive Technologies .
40.
Wright, B.D. and Masters, G.N.1982: Rating scale analysis. Chicago : MESA Press.
