Examining validity in a performance test: the listening summary translation exam (LSTE)

Abstract

This article reports on a project undertaken by the Center for Applied Linguistics (CAL) to develop and validate a criterion-referenced perform ance test of listening summary translation ability for the Federal Bureau of Investigation (FBI).¹ The listening summary translation exam (LSTE- Spanish version) is designed to assess ability to comprehend and summarize in written English recorded conversations spoken in Spanish. The language and topics of these conversations are intended to be representative of the conversations which the FBI routinely monitors.

Bachman (1991) mentions two fundamental requirements for ensuring the validity of effective language testing procedures. First, the language abilities measured by the test must correspond to those abilities needed to carry out tasks in the target-language use situation. Secondly, features of the test tasks, or test method characteristics, must correspond to critical features of target- language use tasks. Bachman (1990; 1991) has suggested that the degree to which a language test meets these requirements can be assessed by evaluating the situational and interactional authenticity of the test tasks.

Although the Bachman framework was not available to us at the time we developed this test (1988-89), we employ the framework here to present the test and to provide evidence of its validity. Evidence of the situational and interactional authenticity of the LSTE-Spanish is offered, as well as for convergent/divergent validity through comparisons of correlations with cri terion measures.

Get full access to this article

View all access options for this article.

References

Bachman, L. 1990: Fundamental considerations in language testing. Oxford: Oxford University Press.

— 1991: What does language testing have to offer? TESOL Quarterly 25, 671-704.

Bernhardt, E.B. and Deville, C. 1991: Testing in foreign language programs and testing programs in foreign language departments: reflections and recommendations. In Teschner, R.V. , editor, Assessing foreign language proficiency of undergraduates, Boston, MA: Heinle & Heinle, 43-59.

Cohen, A.D. 1989: The taking and rating of summary tasks. Paper presented at the 11th Language Testing Research Colloquium , San Antonio, 4-6 March.

Dunkel, P. 1988: The content of L1 and L2 students' lecture notes and its relation to test performance. TESOL Quarterly 22, 259-81.

Joiner, E.G. 1991: Teaching listening: ends and means. In Georgetown University round table on languages and linguistics 1991, Washington, DC: Georgetown University Press, 194-214.

Lund, R. 1990: A taxonomy for teaching second language listening. Foreign Language Annals 23, 105-15.

— 1991: A comparison of L2 listening and reading comprehension . Modern Language Journal 75, 196-204.

Stansfield, C.W. , Scott, M.L. and Kenyon, D.M. 1990: Listening summary translation exam (LSTE) - Spanish. Final project report, revised. Washington, DC: Center for Applied Linguistics, and Springfield, VA : ERIC Document Reproduction Service, ED 323 786.

10.

— 1992: The measurement of translation ability. Modern Language Journal 76, 455-67.

11.

Taylor, S.V. 1994: The effect of varying the immediate recall protocol procedure on recall measures of listening comprehension. Paper presented at the 16th Annual Language Testing Research Colloquium (LTRC) , Washington, DC, 7 March.

12.

Ur, P. 1984: Teaching listening comprehension. Cambridge: Cambridge University Press.

Examining validity in a performance test: the listening summary translation exam (LSTE) - Spanish version

Abstract

Get full access to this article

References