Validation of listening comprehension tests using item response theory

Abstract

This paper presents evidence for the construct validity of foreign language listening comprehension tests obtained from native speaker data and target population data on the tests. Data are analysed using the Rasch model and item parameters estimated using conditional maximum likelihood estimation. A statistical test is used to identify causes for lack of fit of the data with the Rasch model, and an iterative item deletion procedure is proposed to improve Rasch homogeneity of tests. The measure of fit of items in the analysis of item responses from the target population is shown to be associated with native speakers' results on these items. It is suggested that, as the underlying trait of tests for foreign language listening comprehension, literal understanding of aurally perceived language is preferable to interpretative understanding.