What does the analysis of C-test gaps tell us about the construct of a C-test? A comparison of foreign and heritage language learners’ performance

Abstract

In view of the ubiquitous increase in the use of C-tests, which are almost unanimously believed to measure general language proficiency, this study investigates whether the aspects of language proficiency tapped into by the C-test format are the same when the test is taken by a learner population other than that of foreign language learners. Specifically, we conducted a differential functioning analysis and compared the types of mistakes that 113 foreign language learners of Russian made when completing C-test gaps, with the performance of 89 heritage language learners on the same C-test. The results showed that almost half of the C-test gaps are biased towards either learner group. In addition, the error analysis for a number of the biased items demonstrated that, although heritage language learners seem to have an advantage in reconstructing the meaning of C-test gaps, they fail to translate their recognition skills into producing the right form. Furthermore, the study reveals a possible sensitivity of the C-test construct to the traditionally used dichotomous scoring method. We conclude with a discussion that includes the implications of the results regarding the construct measured by the C-test and the possible consequences for its actual use.