Based on and synthesizing recent advances in both psychometric procedures and communicatively oriented linguistic analysis, a theoretical framework for describing factors affecting performance on foreign/second language tests is presented. Included within this framework are both test method factors—such as the testing environment, the nature of the instructions to the examinee, and the stimulus and response modalities represented in the test—and linguistic factors, including the examinee's organizational and pragmatic competence, strategic competence, and psychophysiological skills involved in the proper reception or production of the test language. Implications of the framework for the development and validation of communicative language proficiency tests are discussed and an action plan is suggested, including further refinement of the theoretical framework; development of large-scale, highly authentic criterion measures operationalizing each of the framework factors; and subsequent validation of both existing and to-be-developed practically oriented communicative proficiency tests against the criterion measures. Establishment of a working group of interested individuals from several institutions and disciplinary areas is proposed as an appropriate administrative vehicle for these activities.
Get full access to this article
View all access options for this article.
References
1.
1. We will use the term “ability” to refer both to the knowledge, or competence, involved in language use and to the skill in implementing that knowledge, and the term “language use” to refer to both productive and receptive performance.
2.
Given the problems of clearly delineating real-life from non-real-life language use, however, we prefer Widdowson's notion of authenticity, which he defines essentially as the interaction between the language user and the discourse, in which the language user responds to and interprets both the propositional and illocutionary meaning. Henry G. Widdowson, Teaching Language as Communication (Oxford: Oxford University Press, 1978).
3.
3. John B. Carroll, “A Factor Analysis of Verbal Abilities,”Psychometrika, 6:279-307 (1941).
4.
idem , Language Tests at School: A Pragmatic Approach (London: Longman, 1979).
5.
H. J. Vollmer and F. Sang, “Competing Hypotheses about Second Language Ability: A Plea for Caution,” in ibid., pp. 29-74.
6.
John A. Upshur and Taco J. Homburg, “Some Relations among Language Tests at Successive Ability Levels,” in Issues in Language Testing Research, ed. Oller, pp. 188-201.
7.
Patrick E. Griffin , “The Use of Latent Trait Models in the Calibration of Tests of Spoken Language in Large-scale Selection-placement Programs,” in New Directions in Language Testing, ed. Y. P. Leeet al. (Oxford: Pergamon Press, 1985), pp. 163-170.
8.
8. For example, Robert E. Bolus, Frances B. Hinofotis, and Kathleen M. Bailey, “An Introduction to Generalizability Theory in Second Language Research,”Language Learning, 32(2):245-258 (June 1982).
9.
9. For example, Thom Hudson and Brian Lynch, “A Criterion-referenced Measurement Approach to ESL,”Language Testing, 1(2):171-201 (June 1984).
10.
10. Many of these developments are represented in the papers included in Stansfield, ed., Technology and Language Testing.
11.
John B. Carroll , “Fundamental Considerations in Testing English Proficiency of Foreign Students,” in Testing the English Proficiency of Foreign Students (Washington, DC: Center for Applied Linguistics, 1961), pp. 31-40.
12.
12. Carroll discusses “linguistic performance abilities,” such as speed and diversity of response, complexity of information processing, and awareness of linguistic competence, but considers these as essentially outside the construct of language proficiency. See John B. Carroll, “The Psychology of Language Testing,” in Language Testing Symposium: A Psycholinguistic Approach, ed. Alan Davies (Oxford: Oxford University Press, 1968), pp. 46-69.
13.
Sandra J. Savignon , Communicative Competence: Theory and Classroom Practice (Reading, MA: Addison-Wesley, 1983).
14.
14. Bachman, Fundamental Considerations in Language Testing.
15.
Susanna M. Brütsch, “Convergent-discriminant Validation of Prospective Teacher Proficiency in Oral and Written French by Means of the MLA Cooperative Language Proficiency Test, French Direct Proficiency Tests for Teachers (TOP and TWP), and Self-ratings” (Ph.D. diss., University of Minnesota, 1979); Bachman and Palmer, “Construct Validation of the FSI Oral Interview”; idem, “Construct Validation of Some Components of Communicative Proficiency.”
16.
16. Bachman, Fundamental Considerations in Language Testing.
17.
17. The extent to which a given test or test item needs to be authentic will depend, to a large degree, upon its purpose. Thus, if the purpose of the test is to measure grammatical competence, the need for authenticity may be less than if sociolinguistic or illocutionary competence is to be measured. The effect of authenticity or lack of authenticity on test performance is, of course, an empirical question and is the proper object of construct validation research.
18.
18. We would note that it is possible for a given test to be both criterion and norm referenced, since a test developed by CR procedures could be normed to various groups of test takers. Criterion referencing a norm-referenced test is much more difficult, since it depends on the availability of a well-defined domain of abilities and involves statistical assumptions in score equating. An example of this latter procedure is provided in John B. Carroll, The Foreign Language Attainments of Language Majors in Their Senior Year: A Survey Conducted on U.S. Colleges and Universities. Final Report, ERIC document ED 013-343 (Cambridge, MA: Harvard University Graduate School of Education, 1967).
19.
John L. D. Clark , “Toward a Common Measure of Speaking Proficiency,” in Measuring Spoken Language Proficiency, ed. J. R. Firth (Washington, DC: Georgetown University Press, 1980).
20.
20. American Council on the Teaching of Foreign Languages, ACTFL Proficiency Guidelines (New York: American Council on the Teaching of Foreign Languages, 1986), p. 2.
21.
21. Glaser states that “the notion of a continuum of knowledge acquisition ranging from no proficiency at all to perfect performance” underlies CR measures. Robert Glaser, “Instructional Technology and the Measurement of Learning Outcomes,”American Psychologist, 18(8):519 (Aug. 1963).
22.
James Lantolf and William Frawley, “Oral Proficiency Testing: A Critical Analysis,”Modern Language Journal, 69:337-345 (1985).
23.
23. Variation in native norms is discussed in Braj B. Kachru, “Standards, Codification and Sociolinguistic Realism: The English Language in the Outer Circle,” in English in the World: Teaching and Learning the Language and Literature, ed. R. Quirk and H. G. Widdowson (Cambridge: Cambridge University Press, 1985), pp. 11-30.
24.
24. Clark, “Toward a Common Measure of Speaking Proficiency.”
25.
Howard E. Sollenberger , “Development and Current Use of the FSI Oral Interview Test,” in Direct Testing of Speaking Proficiency, ed. Clark, pp. 1-12.
26.
26. P. Lowe, ILR Handbook on Oral Interview Testing (Washington, DC: DLI/LS Oral Interview Project, 1985).
27.
Pardee Lowe , “The ILR Proficiency Scale as a Synthesizing Research Principle: The View from the Mountain,” in Foreign Language Proficiency in the Classroom and Beyond, ed. Charles J. James (Lincolnwood, IL: National Textbook, 1985).
28.
28. Pardee Lowe and Ray T. Clifford, “Developing an Indirect Measure of Overall Oral Proficiency,” in Measuring Spoken Language Proficiency, ed. Firth, pp. 31-39.
29.
29. John L. D. Clark, “Development of a Tape-mediated, ACTFL/ILR Scale-based Test of Chinese Speaking Proficiency,” in Technology and Language Testing, ed. Stansfield, pp. 129-146.
30.
30. Educational Testing Service, Japanese Proficiency Test (Princeton, NJ: Educational Testing Service, 1981).
31.
31. Center for Applied Linguistics, Chinese Proficiency Test (Washington, DC: Center for Applied Linguistics, 1983).
32.
32. Educational Testing Service, Russian Proficiency Test (Princeton, NJ: Educational Testing Service, in preparation).
33.
33. John L. D. Clark, Final Report: The Development, Validation and Dissemination of a Proficiency-based Test of Speaking Ability in Chinese and Associated Assessment Model for Other Less-commonly Taught Languages (Washington, DC: Center for Applied Linguistics, forthcoming).
