Within the fields of applied linguistics and language testing, there has been a recent interest in rating scales, and how rating scales are constructed (Upshur and Turner, 1995). This is not surprising, as there is increasing concern that scores from language tests should be meaningful in applied linguistics terms. However, applied linguistics research and second language acquisition research have done little to provide descriptions of language abilities or performances which can be operationalized by language testers. Many existing descriptors for bands in rating scales are therefore barely tenable as definitions of constructs.
This article looks at the definition of fluency in the literature, and proposes a qualitative and quantitative approach which may be used to produce a 'thick' description of language use, which can be used in rating scale construction. A fluency rating scale is described, and its reliability and validity assessed. The arti cle suggests that validity considerations must be addressed in the construction phase of developing rating scales, through the careful consideration of the linguis tic meaning of constructs, rather than merely as a post hoc enterprise.
Get full access to this article
View all access options for this article.
References
1.
Adams, M.L.1980: Five cooccuring factors in speaking proficiency. In Frith, J.R., editor, Measuring spoken language proficiency. Washington, DC: Georgetown University Press, 1-6.
2.
Alderson, J.C.1991: Bands and scores. In Alderson, J.C. and North, B., editors, Language testing in the 1990s, London: Modern English Publications and the British Council .
3.
Allwright, D.1988: Observation in the language classroom. London: Longman.
4.
Bachman, L.F.1990: Fundamental considerations in language testing. Oxford: Oxford University Press.
5.
Bachman, L.F. and Clark, J.L.D.1988: The measurement of foreign/second language proficiency . Annals of the American Academy of Political and Social Science490, 20-33.
6.
Bachman, L.F. and Palmer, A.S.1982: The construct validation of some components of communicative proficiency. TESOL Quarterly16, 449-65.
7.
Bachman, L.F. and Savignon, S.J.1986: The evaluation of communicative language proficiency: a critique of the ACTFL Oral Interview. Modern Language Journal70, 380-90.
8.
Bhatia, V.K.1993: Analysing genre: language use in professional settings . London: Longman.
9.
Bialystok, E. and Sharwood Smith, M.1985: Interlanguage is not a state of mind: an evaluation of the construct for second language acquisition . Applied Linguistics6, 101-17.
10.
Boldt, R.F.1991: Second language constructs as indexed by ACTFL ratings and TOEFL scores. Paper presented at the Language Testing Research Colloquium, Princeton, NJ.
11.
Bolus, R.E., Hinofotis, F. and Bailey, K.M.1981: An introduction to generalizability theory in second language research. Language Learning32, 245-58.
12.
Brown, G. and Yule, G.1983: Discourse analysis. Cambridge: Cambridge University Press.
13.
Brumfit, C.1984: Communicative methodology in language teaching: the roles of fluency and accuracy. Cambridge: Cambridge University Press.
14.
Campbell, D.T. and Fiske, D.W.1959: Convergent and discriminant validation by the multitrait-multimethod matrix. PsychologicalBulletin56, 81-105.
15.
Channell, J.1985: Vagueness as a conversational strategy. Nottingham LinguisticCircular14, 3-24.
16.
Chaudron, C.1988: Second language classrooms: research on teaching and learning . Cambridge: Cambridge University Press.
17.
Clahsen, H.1985: Profiling second language development: a procedure for assessing L2 proficiency. In Hyltenstam, K. and Pienemann, M., editors, Modeling and assessing second language acquisition, San Diego, CA: College Hill Press.
18.
Clark, J.L.D. and Clifford, R.T.1988: The FSI/ILR/ACTFL proficiency scales and testing techniques: development, current status and needed research. Studies in Second Language Acquisition10, 129-47.
19.
Crocker, L. and Algina, J.1986: Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
20.
Cronbach, L.J.1984: Essentials of psychological testing. New York: Harper & Row.
21.
Crystal, D. and Davy, D.1985: Advanced conversational English. London : Longman.
22.
Davies, A.1987: Certificate of Proficiency in English. In Alderson, J.C., Krahnke, K. and Stansfield, C.W., editors, Reviews of English language proficiency tests, Washington, DC: TESOL.
23.
Douglas, D.1994: Quantity and quality in speaking test performance. Language Testing11, 125-44.
24.
Dulay, H., Burt, M. and Krashen, S.1982: Language two. Oxford: Oxford University Press.
25.
Ellis, R.1985: Understanding second language acquisition. Oxford: Oxford University Press.
26.
— 1990: Individual learning styles in classroom second language development. In de Jong, J.H.A.L. and Stevenson, K., editors, Individualizing the assessment of language abilities, Philadelphia, PA : Multilingual Matters.
27.
Feldt, L.S. and Brennan, R.L.1989: Reliability. In Linn, R.L., editor, Educational measurement, New York: American Council on Education, Macmillan.
28.
Frith, J.R.1979: Preface. In Adams, M.L. and Frith, J.R., editors, Testing kit: French and Spanish. Washington, DC: Department of State Foreign Services Institute.
29.
Fulcher, G.1987: Tests of oral performance: the need for data-based criteria . English Language Teaching Journal41, 287-91.
30.
— 1988: Lexis and reality in oral testing. Washington, DC: ERIC Clearinghouse on Languages and Linguistics (document reference: ED 298 759).
31.
— 1993: The construction and validation of rating scales for oral tests in English as a foreign language. Unpublished PhD, University of Lancaster.
32.
— 1994: Some priority areas for research in oral language testing . Language Testing Update15, 39-47.
33.
— 1996: Testing tasks: issues in task design and the group oral . Language Testing13, 23-51.
34.
Gaies, S.J.1977: The nature of linguistic input in formal second language learning: linguistic and communicative strategies in ESL teachers' classroom language. In Brown, H.D., Yorio, C.A. and Crymes, R.H. , editors, Teaching and learning English as a second language: trends in research and practice. Selected papers from the 1977 TESOL convention , Washington, DC: TESOL.
35.
Geertz, C.1973: Thick description: toward an interpretative theory of culture. In Geertz, C. The interpretation of cultures: selected essays, New York: Basic Books.
36.
Glaser, B. and Strauss, A.L.1967: The discovery of Grounded Theory: strategies for qualitative research. Chicago; IL: Aldine.
37.
Griffin, P.E.1985: The use of latent trait models in the calibration of tests of spoken language in large-scale selection-placement programs. In Lee, Y.P., editor, New directions in language testing, Oxford: Pergamon Institute of English.
38.
Griffiths, R.1991: Pausological research in an L2 context: a rationale, and review of selected studies. Applied Linguistics12, 345-64.
39.
Grosjean, F.1980: Temporal variables within and between languages. In Dechert, H. and Raupach, M., editors, Towards a cross-linguistic assessment of speech production, Frankfurt: Peter Lang.
40.
Hamp-Lyons, L.1987: Cambridge First Certificate in English. In Alderson, J.C., Krahnke, K. and Stansfield, C.W., editors, Reviews of English language proficiency tests, Washington, DC: TESOL.
41.
Hatch, E.1974: Second language universals. Working Papers onBilingualism3, 1-17.
42.
Hedge, T.1993: Key concepts in ELT: fluency. English Language Teaching Journal47, 275-77.
Hieke, A.E.1985: A componential approach to oral fluency evaluation. Modern Language Journal69, 135-42.
45.
Ingram, D.1982: Introduction to the ASLPR. Mimeo, Brisbane College of Advanced Education.
46.
Jarvis, G.A.1986: Proficiency testing: a matter of false hopes?ADFL Bulletin18, 20-21.
47.
Joreskog, K.G.1969: A general approach to confirmatory maximum likelihood factor analysis. Psychometrika34, 183-202.
48.
Klippel, F.1984: Keep talking: communicative fluency activities for language teaching. Cambridge: Cambridge University Press.
49.
Larsen-Freeman, D. and Long, M.1991: An introduction to second language acquisition research . London: Longman.
50.
Lazaraton, A.1992: The structural organization of a language interview: a conversation analytic perspective. System20, 373-86.
51.
Long, M.1983: Inside the 'black box': methodological issues in classroom research on language learning. In Seliger, H.W. and Long, M., editors, Classroom oriented research in second language acquisition, Rowley, MA: Newbury House.
52.
Manning, P.K.1991: Analytic induction. In Plumber, K., editor, Symbolic Interactionism. Vol. 2. Contemporary issues. Brookfield, VT: Edward Elgar , 401-30.
53.
Meisel, J.M.1980: Linguistic simplification. In Felix, S., editor, Second language development: trends and issues, Tubingen: Gunter Narr.
54.
Meredith, R.A.1978: Improved oral test scores through delayed response. Modern Language Journal62, 321-27.
55.
Messick, S.1989: Validity. In Linn, L. , editor, Educational measurement. New York: American Council on Education/ Macmillan.
56.
North, B.1994a: Scales of language proficiency: a survey of some existing systems. Washington, DC: Georgetown University Press.
57.
— 1994b: Item and descriptor-banking: objective underpinning of subjective assessment in schools. Paper presented at the Language Testing Forum, University of Cambridge Local Examinations Syndicate, 11 December.
58.
Perrett, G.1990: The language testing interview: a reappraisal. In de Jong, J. and Stevenson, D.K., editors, Individualizing the assessment of language abilities, Philadelphia, PA: Multilingual Matters.
59.
Rixon, S.1992: English and other languages for younger children: practice and theory in a rapidly changing world. Language Teaching25, 73-93. Sacks, H., Schegloff, E.A. and Jefferson , G.1974: A simplest systematics for the organization of turn-taking for conversation . Language50, 696-735.
60.
Seliger, H.W.1980: Utterance planning and correction behaviour: its function in the grammar construction process for second language learners. In Dechert, H. and Raupach, M., editors. Towards a cross-linguistic assessment of speech production, Frankfurt: Peter Lang.
61.
Shohamy, E.1990: Discourse analysis in language testing. Annual Review of Applied Linguistics11, 115-28.
62.
Sollenberger, H.E.1978: Development and current use of the FSI Oral Interview Test. In Clark, J.L.D., editor, Direct testing of speaking proficiency: theory and application, Princeton, NJ: Educational Testing Service.
63.
Strauss, A. and Corbin, J.1994: Grounded theory methodology: an overview. In Denzin, N.K. and Lincoln, Y.S., editors, Handbook of qualitative research, London: Sage.
64.
Swales, J.1990: Genre analysis. Cambridge: Cambridge University Press.
65.
Upshur, J.A. and Turner, C.E.1995: Constructing rating scales for second language tests. English Language Teaching Journal49, 3-12.
66.
Weir, C.1987: English Language Testing Service. In Alderson, J.C., Krahnke, K. and Stansfield, C.W., editors, Reviews of English language proficiency tests, Washington, DC: TESOL.
67.
Wilkinson, A.1988: Systat: the system for statistics. Evanston, IL: Systat Inc.
68.
Winter, E.O.1978: A look at the role of certain words in information structure . In Jones, K.P. and Horsnell, V., editors, Informatics 3, London: ASLIB.
