This study aimed to propose a new method for scoring C-Tests as measures of general language proficiency. In this approach, the unit of analysis is sentences rather than gaps or passages. That is, the gaps correctly reformulated in each sentence were aggregated as sentence score, and then each sentence was entered into the analysis as a polytomous item. Rasch partial credit model was applied to analyze the test. To investigate the effectiveness of the new method, the results of this new strategy were compared with those derived from passage and gap levels as well as combining locally dependent items using the dichotomous Rasch model, the rating scale model, and the partial credit model. The obtained scores from the administration of a C-Test comprising four English passages, each containing 25 gaps, to 160 participants were subjected to dichotomous and polytomous Rasch model analyses. The models were compared regarding individual item fit, person/item separation and reliability, global fit (e.g., deviance), unidimensionality, local item dependence, and person parameters. Results showed the effectiveness of the new scoring method. The suggested strategy increases the number of items compared to the super-item approach, provides information at the sentence level, and reduces the impact of local independence.
AldersonJ. C. (1979). The cloze procedure and proficiency in English as a foreign language. TESOL Quarterly, 13(2), 219–227. https://doi.org/10.2307/3586211
BabaiiE.Fatahi-MajdM. (2014). Failed restorations in the C-test: Types, sources, and implications for C-test processing. In GrotjahnR. (Ed.), Der C-Test: Aktuelle Tendenzen [The C-Test: Current trends] (pp. 263–276). Peter Lang.
6.
BabaiiE.Jalali MoghaddamM. (2006). On the interplay between test task difficulty and macro-level processing in the C-test. System, 34(4), 586–600. https://doi.org/10.1016/j.system.2006.09.002
7.
BabaiiE.ShahriS. (2010). Psychometric rivalry: The C-test and the cloze test interacting with test takers’ characteristics. In GrotjahnR. (Ed.), Der C-Test: Beiträge aus der aktuellen Forschung [The C-Test: Contributions from current research] (pp. 41–56). Peter Lang.
BaghaeiP.ChristensenK. B. (2023). Modeling local item dependence in C-tests with the loglinear Rasch model. Language Testing, 40(3), 820–827. https://doi.org/10.1177/02655322231155109
10.
BaghaeiP.EffatpanahF. (2023). An alternative strategy for modelling local item dependence in C-tests. In DobricN.CesnikH.HarschC. (Eds.), Festschrift in honour of Guenther Sigott: Advanced methods in language testing (pp. 153–164). Peter Lang.
11.
BaghaeiP.Tabatabaee-YazdiM. (2015). The C-test: An integrative measure of crystallized intelligence. Journal of Intelligence, 3(2), 46–58. https://doi.org/10.3390/jintelligence3020046
12.
BirnbaumA. (1968). Some latent trait models and their use in inferring an examinee’s ability. In LordF. M.NovickM. R. (Eds.), Statistical theories of mental test scores (pp. 397–479). Addison-Wesley.
13.
BradlowE. T.WainerH.WangX. (1999). A Bayesian random effects model for testlets. Psychometrika, 64(2), 153–168. https://doi.org/10.1007/BF02294533
ChristensenK. B.MakranskyG.HortonM. (2017). Critical values for Yen’s Q3: Identification of local dependence in the Rasch model using residual correlations. Applied Psychological Measurement, 41(3), 178–194. https://doi.org/10.1177/0146621616677520
DrackertA.TimukovaA. (2020). What does the analysis of C-test gaps tell us about the construct of a C-test? A comparison of foreign and heritage language learners’ performance. Language Testing, 37(1), 107–132. https://doi.org/10.1177/0265532219861042
19.
EckesT. (2010). Rasch models for C-tests: Closing the gap on modern psychometric theory. In BerndtA.KleppinK. (Eds.), Sprachlehrforschung: Theorie und Empirie—Festschrift für Rüdiger Grotjahn [Language teaching research: Theoretical and empirical foundations—Festschrift for Rüdiger Grotjahn] (pp. 39–49). Peter Lang.
EckesT.BaghaeiP. (2015). Using testlet response theory to examine local dependence in C-tests. Applied Measurement in Education, 28(2), 85–98. https://doi.org/10.1080/08957347.2014.1002919
EffatpanahF.BaghaeiP. (2022). Evaluating different scoring methods for the speeded cloze-elide test: The application of the Rasch partial credit model. The Quantitative Methods for Psychology, 18(3), 241–254. https://doi.org/10.20982/tqmp.18.3.p241
24.
ForthmannB.GrotjahnR.DoeblerP.BaghaeiP. (2020). A comparison of different item response theory models for scaling speeded C-Tests. Journal of Psychoeducational Assessment, 38(6), 692–705. https://doi.org/10.1177/0734282919889262
25.
GermannU.GrotjahnR. (1994). Das lösen von C-tests auf dem computer: Eine pilotuntersuchung zu den bearbeitungsprozessen. In GrotjahnR (Ed.), Der C-Test: Theoretische Grundlagen und praktische Anwendungen [The C-test: Theoretical foundations and practical applications] (Vol. 2, pp. 279–304). Brockmeyer.
GrotjahnR. (1987). How to construct and evaluate a C-Test: A discussion of some problems and some statistical analyses. In GrotjahnR.Klein-BraleyC.StevensonD. K. (Eds.), Taking their measure: The validity and validation of language tests (pp. 219–253). Brockmeyer.
28.
GrotjahnR. (1992). Der C-Test im Französischen: Quantitative Analysen. In GrotjahnR. (Ed.), Der C-Test. Theoretische Grundlagen und praktische Anwendungen [The C-test: Theoretical foundations and practical applications] (Vol. 1, pp. 205–255). Brockmeyer.
29.
GrotjahnR. (1996). Einleitende Bemerkungen zu band 3 [Introductory notes on volume 3]. In GrotjahnR. (Ed.), Der C-Test: Theoretische Grundlagen und praktische Anwendungen [The C-test: Theoretical foundations and practical applications] (Vol. 3, pp. 1–22). Brockmeyer.
30.
GrotjahnR. (2002). “Scrambled” C-Tests: Eine Folgeuntersuchung [“Scrambled” C-tests: A follow-up study]. In GrotjahnR. (Ed.), Der C-Test: Theoretische Grundlagen und praktische Anwendungen [The C-test: Theoretical foundations and practical applications] (Vol. 4, pp. 83–121). AKS-Verlag.
31.
GrotjahnR. (2019). C-tests. In JeukS.SettinieriJ. (Eds.), Sprachdiagnostik Deutsch als Zweitsprache: Ein handbuch (pp. 579–603). De Gruyter Mouton.
32.
GrotjahnR.AllnerB. (1996). Der C-Test in der sprachlichen Aufnahmeprüfung an Studienkollegs für ausländische Studierende an Universitäten in Nordrhein-Westfalen. In GrotjahnR. (Ed.), Der C-Test: Theoretische Grundlagen und praktische Anwendungen [The C-test: Theoretical foundations and practical applications] (Vol. 3, pp. 279–342). Brockmeyer.
33.
GrotjahnR.StemmerB. (2002). C-Tests and language processing. In ColemanJ. A.GrotjahnR.RaatzU. (Eds.), University language testing and the C-Test (pp. 115–130). AKS-Verlag.
34.
GrotjahnR.Klein-BraleyC.RaatzU. (2002). C-tests: An overview. In ColemanJ.GrotjahnR.RaatzU. (Eds.), University language testing and the C-Test (pp. 93–114). AKS-Verlag.
35.
HambletonR. K.SwaminathanH. (1985). Item response theory: Principles and applications. Kluwer-Nijhoff.
36.
HarschC.HartigJ. (2010). Empirische und inhaltliche Analyse lokaler Abhängigkeiten im C-Test [Empirical and content analysis of local dependencies in C-tests]. In GrotjahnR. (Ed.), Der C-Test: Beiträge aus der aktuellen Forschung [The C-test: Contributions from current research] (pp. 193–204). Peter Lang.
37.
Klein-BraleyC. (1981). Empirical investigations of cloze tests [Unpublished doctoral thesis, University of Duisburg-Essen].
38.
Klein-BraleyC. (1996a). Hunting unicorns: C-Tests, cloze tests, and the RIP (Removal of Information Procedure). In GrotjahnR. (Ed.), Der C-Test: Theoretische Grundlagen und praktische Anwendungen [The C-test: Theoretical foundations and practical applications] (Vol. 3, pp. 127–137). Brockmeyer.
39.
Klein-BraleyC. (1996b). Towards a theory of C-Test processing. In GrotjahnR. (Ed.), Der C-Test: Theoretische Grundlagen und praktische Anwendungen [The C-test: Theoretical foundations and practical applications] (Vol. 3, pp. 23–94). Brockmeyer.
LinacreJ. M. (1999). Investigating rating scale category utility. Journal of Outcome Measurement, 3, 103–122. http://jampress.org/JOM.htm
42.
LinacreJ. M. (2002). What do infit and outfit, mean-square and standardized mean?Rasch Measurement Transactions, 16(2), 878. https://www.rasch.org/rmt/rmt162f.htm
43.
LinacreJ. M. (2009). A user’s guide to WINSTEPS. Winsteps.
McKayT. H.TeimouriY.SağdiҫA.SalenB.ReaganD.MaloneM. (2021). The cagey C-test construct: Some evidence from a meta-analysis of correlation coefficients. System, 99, 1–19. https://doi.org/10.1016/j.system.2021.102526
46.
MotallebzadehZ. (2023). A comparison of different methods for investigating the reliability of C-Tests. Educational Methods & Practice, 1: 1. https://emp-open.de/Full_text?article_id=45
NadriM.BaghaeiP.ZohoorianZ. (2019). The contribution of cognitive abilities and general language proficiency to explaining listening comprehension in English as a foreign language. Cogent Education, 6(1), 1567010. https://doi.org/10.1080/2331186X.2019.1567010
49.
NorrisJ. M. (2018). Developing and investigating C-tests in eight languages: Measuring proficiency for research purposes. In NorrisJ. M. (Ed.), Developing C-tests for estimating proficiency in foreign language research (pp. 7–33). Peter Lang.
50.
OllerJ. W.Jr (1979). Language tests at school: A pragmatic approach. Longman.
RaatzU. (1984). The factorial validity of C-Tests. In CulhaneT.Klein-BraleyC.StevensonD. K. (Eds.), Practice and problems in language testing 7. Proceedings of the seventh international language testing symposium of the IUS, Colchester, October 1983 (pp. 124–139). Department of Language and Linguistics, University of Essex.
53.
RaatzU.Klein-BraleyC. (1981). The C-test: A modification of the cloze procedure. In CulhaneT.Klein-BraleyC.StevensonD. K. (Eds.), Practice and problems in language testing (pp. 113–145). University of Essex.
54.
RaschG. (1960/1980). Probabilistic models for some intelligence and attainment tests (Expanded Ed.). University of Chicago Press.
SchnoorB.HartigJ.KlingerT.NaumannA.UsanovaI. (2023). Measuring the development of general language skills in English as a foreign language: Longitudinal invariance of the C-test. Language Testing, 40(3), 796–819. https://doi.org/10.1177/02655322231159829
57.
SchroedersU.RobitzschA.SchipolowskiS. (2014). A comparison of different psychometric approaches to modeling testlet structures: An example with C-tests. Journal of Educational Measurement, 51(4), 400–418. https://doi.org/10.1111/jedm.12054
58.
ShoahosseiniR.BaghaeiP.KhodabakhshzadehH.AshrafH. (2024). C-Test construct validity: Evidence from nonparametric item response theory. Language Testing in Asia, 14(10), 1–15. https://doi.org/10.1186/s40468-024-00279-z
59.
SigottG. (2004). Towards identifying the C-Test construct. Peter Lang.
SpolskyB. (1969, September8–12). Reduced redundancy as a language testing tool [Paper presentation]. Second International Congress of Applied Linguistics, Cambridge, England.
62.
SpolskyB. (1971). Reduced redundancy as a language testing tool. In PerrenG. E.TrimJ. L. M. (Eds.), Applications of linguistics (pp. 383–390). Cambridge University Press.
63.
StemlerS. E.NaplesA. (2021). Rasch measurement v. item response theory: Knowing when to cross the line. Practical Assessment, Research and Evaluation, 26(11), 1–16. https://doi.org/10.7275/v2gd-4441
64.
StemmerB. (1991). What’s on a C-Test taker’s mind: Mental processes in C-Test taking. Brockmeyer.
65.
WainerH.BradlowE. T.WangX. (2007). Testlet response theory and its applications. Cambridge University Press.
WrightB. D. (1994). Local dependency, correlations, and principal components. Rasch Measurement Transactions, 10(3), 509–511. https://www.rasch.org/rmt/rmt103b.htm
YenW. M.FitzpatrickA. R. (2006). Item response theory. In BrennanR. L. (Ed.), Educational measurement (4th ed., pp. 111–153). American Council on Education/Praeger.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
