What C-tests actually measure has been an issue of debate for many years. In the
present research, the authors examined the hypothesis that C-tests measure general
language proficiency. A total of 843 participants from four independent samples took
a German C-test along with the TestDaF (Test of German as a Foreign Language). Rasch
measurement modelling and confirmatory factor analysis provided clear evidence that
the C-test in question was a highly reliable, unidimensional instrument, which
measured the same general dimension as the four TestDaF sections: reading,
listening, writing and speaking. Moreover, the authors showed that language
proficiency was divisible into more specific constructs and that examinee
proficiency level differentially influenced C-test performance. The findings have
implications for the multicomponentiality and fluidity of the C-test measurement construct.
Get full access to this article
View all access options for this article.
References
1.
Alderson, J. C.2002: Testing proficiency and achievement: principles and
practice. In
Coleman, J. A.
,
Grotjahn, R.
and
Raatz, U.
, editors, University language testing and the C-Test.
Bochum: AKS-Verlag
, 15-30.
2.
Alderson, J. C.
,
Clapham, C.
and
Steel, D.1997:
Metalinguistic knowledge, language aptitude and language proficiency
. Language Teaching Research1,
93-121
.
3.
Andrich, D.1978:
A rating formulation for ordered response categories
. Psychometrika43,
561-573
.
4.
Arras, U.
,
Eckes, T.
and
Grotjahn, R.2002: C-Tests im Rahmen des ‘Test Deutsch als
Fremdsprache’ (TestDaF): Erste Forschungsergebnisse [C-tests within
the ‘Test of German as a foreign language’ (TestDaF):
preliminary research findings]. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
4.
Bochum: AKS-Verlag
, 175-209.
5.
Babaii, E.
and
Ansary, H.2001:
The C-test: a valid operationalization of reduced redundancy principle?System29,
209-219
.
6.
Bachman, L. F.1990: Fundamental considerations in language testing.
Oxford: Oxford University Press
.
7.
Bachman, L. F.2000:
Modern language testing at the turn of the century: assuring that what we
count counts
. Language Testing17,
1-42
.
8.
Bachman, L. F.2004: Statistical analyses for language assessment.
Cambridge: Cambridge University Press
.
9.
Bachman, L. F.
,
Davidson, F.
,
Ryan, K.
and
Choi, I.-C.1995: An investigation into the comparability of two tests of
English as a foreign language: the Cambridge-TOEFL comparability study.
Cambridge: Cambridge University Press
.
10.
Bachman, L. F.
and
Palmer, A. S.1996: Language testing in practice: designing and developing useful
language tests.
Oxford: Oxford University Press
.
11.
Bae, J.
and
Bachman, L. F.1998:
A latent variable approach to listening and reading: testing factorial
invariance across two groups of children in the Korean/English two-way
immersion program
. Language Testing15,
380-414
.
12.
Baker, R.1997: Classical test theory and item response theory in test
analysis.
Lancaster: Centre for Research in Language
Education, Lancaster University
.
13.
Bejar, I. I.1983: Achievement testing: recent advances.
Beverly Hills, CA: Sage
.
14.
Bond, T. G.
and
Fox, C. M.2001: Applying the Rasch model: fundamental measurement in the human
sciences.
Mahwah, NJ: Lawrence Erlbaum
.
15.
Boonsathorn, S. 1987: C-Tests, proficiency, and reading strategies in ESL.
Unpublished PhD dissertation, University of Alberta, Canada.
16.
Byrne, B. M.1998: Structural equation modeling with LISREL, PRELIS, and SIMPLIS:
basic concepts, applications, and programming.
Mahwah, NJ: Lawrence Erlbaum
.
17.
Carroll, J. B.1983: Psychometric theory and language testing. In
Oller, J. W. Jr.
, editor, Issues in language testing research.
Rowley, MA: Newbury House
, 80-107.
18.
Review of
Klein-Braley, C.
and
Raatz, U.1985: C-Tests in der Praxis [C-testing in practice].
Bochum: AKS
.
19.
Carroll, J. B.1993: Human cognitive abilities: a survey of factor-analytic
studies.
Cambridge: Cambridge University Press
.
20.
Chapelle, C. A.1994:
Are C-tests valid measures for L2 vocabulary research?Second Language Research10,
157-187
.
21.
Chapelle, C. A.
and
Abraham, R. G.1990:
Cloze method: what difference does it make?Language Testing7,
121-146
.
22.
Chihara, T.
,
Cline, W. D.
and
Sakurai, T.1996: If the cloze test is a question, is the C-test the
answer? In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
3.
Bochum: Brockmeyer
, 183-195.
23.
Clapham, C.1996: The development of IELTS: a study of the effect of background
knowledge on reading comprehension.
Cambridge: Cambridge University Press
.
24.
Cohen, A. D.
,
Segal, M.
and
Weiss Bar-Siman-Tov, R.1985: The C-Test in Hebrew. In
Klein-Braley, C.
and
Raatz, U.
, editors, C-Tests in der Praxis [C-testing in practice].
Bochum: AKS-Verlag
, 121-127.
25.
Daller, H.
and
Grotjahn, R.1999:
The language proficiency of Turkish returnees from Germany: an empirical
investigation of academic and everyday language proficiency
. Language, Culture and Curriculum12,
156-172
.
26.
Daller, H.
and
Phelan, D.2006: The C-test and TOEIC® as measures of
students’ progress in intensive short courses in EFL. In
Grotjahn, R.
, editor, Der C-Test: Theorie, Empirie, Anwendungen/The C-test: theory,
empirical research, applications.
Frankfurt am Main: Peter Lang
, 101-119.
27.
Daller, H.
,
van Hout, R.
and
Treffers-Daller, J.2003:
Lexical richness in the spontaneous speech of bilinguals
. Applied Linguistics24,
197-222
.
28.
Dörnyei, Z.
and
Katona, L.1992:
Validation of the C-test amongst Hungarian EFL learners
. Language Testing9,
187-206
.
29.
Douglas, D.2000: Assessing languages for specific purposes.
Cambridge: Cambridge University Press
.
30.
Eckes, T.2003:
Qualitätssicherung beim TestDaF: Konzepte, Methoden, Ergebnisse
[Quality assurance for TestDaF: concepts, methods, results]
. Fremdsprachen und Hochschule69,
43-68
.
31.
Eckes, T.2004:
Beurteilerübereinstimmung und Beurteilerstrenge: Eine
Multifacetten-Rasch-Analyse von Leistungsbeurteilungen im ‘Test
Deutsch als Fremdsprache’ (TestDaF) [Rater agreement and rater
severity: a many-facet Rasch analysis of performance assessments in the
‘Test of German as a foreign language’ (TestDaF)]
. Diagnostica50,
65-77
.
32.
Eckes, T.2005a:
Evaluation von Beurteilungen: Psychometrische Qualitätssicherung
mit dem Multifacetten-Rasch-Modell [Evaluation of ratings: psychometric
quality assurance via many-facet Rasch measurement]
. Zeitschrift für Psychologie213,
77-96
.
33.
Eckes, T.2005b:
Examining rater effects in TestDaF writing and speaking performance
assessments: a many-facet Rasch analysis
. Language Assessment Quarterly2,
197-221
.
34.
Eckes, T.2006: Rasch-Modelle zur C-Test-Skalierung [Rasch models for
C-tests]. In
Grotjahn, R.
, editor, Der C-Test: Theorie, Empirie, Anwendungen/The C-test: theory,
empirical research, applications.
Frankfurt am Main: Peter Lang
, 1-44.
35.
Eckes, T.
and
Grotjahn, R.2006: C-Tests als Anker für TestDaF: Rasch-Analysen mit
dem kontinuierlichen Ratingskalen-Modell [C-tests as anchor for TestDaF: Rasch
analyses via the continuous rating scale model]. In
Grotjahn, R.
, editor, Der C-Test: Theorie, Empirie, Anwendungen [The C-test: theory,
empirical research, applications].
Frankfurt am Main: Peter Lang
, 167-193.
36.
Eckes, T.
,
Ellis, M.
,
Kalnberzina, V.
,
Pižorn, K.
,
Springer, C.
,
Szollás, K.
and
Tsagari, C.2005:
Progress and problems in reforming public language examiniations in Europe:
cameos from the Baltic States, Greece, Hungary, Poland, Slovenia, France,
and Germany
. Language Testing22,
355-377
.
37.
Paper presented as part of the symposium ‘Research in
support of the development of new TOEFL’ at the meeting of the
National Council on Measurement in Education, New Orleans,
LA, April 2002.
38.
Farhady, H.
and
Jamali, F.1999:
Varieties of C-test as measures of general proficiency
. Journal of the Faculty of Foreign Languages3,
23-42
(
Tehran University Press
).
39.
Fouly, K. A.
,
Bachman, L. F.
and Cziko, G.A. 1990:
The divisibility of language competence: a confirmatory approach
. Language Learning40,
1-21
.
40.
Ginther, A.
and
Stevens, J.
1998:
Language background, ethnicity, and the internal construct validity of the
Advanced Placement Spanish Language Examination
. In
Kunnan, A. J.
, editor, Validation in language assessment: selected papers from the
17th Language Testing Research Colloquium, Long
Beach. Mahwah, NJ: Lawrence Erlbaum,
169-194
.
41.
Grotjahn, R.1987: How to construct and evaluate a C-Test: a discussion of
some problems and some statistical analyses. In
Grotjahn, R.
,
Klein-Braley, C.
and
Stevenson, D. K.
, editors, Taking their measure: the validity and validation of language
tests.
Bochum: Brockmeyer
, 219-253.
42.
Grotjahn, R.1992: Der C-Test im Französischen: quantitative analysen
[Quantitative analyses of a French C-test]. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
1.
Bochum: Brockmeyer
, 205-255.
43.
Grotjahn, R.1995:
Der C-Test: state of the art
. Zeitschrift für Fremdsprachenforschung6,
37-60
.
44.
Grotjahn, R.2002: ‘Scrambled’ C-Tests: eine
Folgeuntersuchung [‘Scrambled’ C-tests: a follow-up
study]. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
4.
Bochum: AKS-Verlag
, 83-121.
45.
Proceedings of the ALTE Barcelona ConferenceJuly 2001.
46.
Grotjahn, R.2006: 25 Jahre C-Test: Einleitung und Übersicht
über den Band [25 years of research into the C-test:
introduction]. In
Grotjahn, R.
, editor, Der C-Test: Theorie, Empirrie, Anwendungen [The C-test: theory,
empirical research, applications].
Frankfurt am Main: Peter Lang
, ix-xxv.
47.
Grotjahn, R.
,
Klein-Braley, C.
and
Raatz, U.2002: C-Tests: an overview. In
Coleman, J. A.
,
Grotjahn, R.
and
Raatz, U.
, editors, University language testing and the C-Test.
Bochum: AKS-Verlag
, 93-114.
48.
Grotjahn, R.
and
Stemmer, B.2002: C-Tests and language processing. In
Coleman, J. A.
,
Grotjahn, R.
and
Raatz, U.
, editors, University language testing and the C-Test.
Bochum: AKS-Verlag
, 115-130.
49.
Grotjahn, R.
and
Tönshoff, W.1992: Textverständnis bei der C-Test-Bearbeitung:
Pilotstudien mit Französisch- und Italienischlernern [Text
comprehension and C-test-taking: pilot studies with learners of French and
Italian]. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
1.
Bochum: Brockmeyer
, 19-95.
50.
Hair, J. F.
,
Anderson, R. E.
,
Tatham, R. L.
and
Black, W. C.1998: Multivariate data analysis. 5th edition.
Upper Saddle River, NJ: Prentice-Hall
.
51.
Hastings, A. J.2002a: Error analysis of an English C-Test: evidence for
integrated processing. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
4.
Bochum: AKS-Verlag
, 53-66.
52.
Hastings, A. J.2002b: In defense of C-testing. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
4.
Bochum: AKS-Verlag
, 11-25.
53.
Henning, G.1992:
Dimensionality and construct validity of language tests
. Language Testing9,
1-11
.
54.
Huhta, A.1996: Validating an EFL C-test for students of English
philology. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
3.
Bochum: Brockmeyer
, 197-234.
55.
Jafarpur, A.2002: A comparative study of a C-Test and a cloze test.
In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
4.
Bochum: AKS-Verlag
, 31-51.
56.
Jakschik, G.1996: Validierung des C-Tests für erwachsene
Zweitsprachler. Eine Längsschnittuntersuchung bei Trägern von
beruflichen Bildungsmaßnahmen [C-test validation for adult second
language learners: a longitudinal study with bodies responsible for vocational
training programmes]. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
3.
Bochum: Brockmeyer
, 235-277.
57.
Jöreskog, K. G.
and
Sörbom, D.1993: LISREL 8: structural equation modeling with the SIMPLIS
command language.
Chicago: Scientific Software International
.
58.
Jöreskog, K. G.
and
Sörbom, D.1996a: LISREL 8: user’s reference guide.
Chicago: Scientific Software International
.
59.
Jöreskog, K. G.
and
Sörbom, D.1996b: PRELIS 2: user’s reference guide.
Chicago: Scientific Software International
.
60.
Kamimoto, T.1992:
An inquiry into what a C-Test measures
. Fukuoka Women’s Junior College Studies44,
67-79
.
61.
Kenyon, D. M.2000: Tape-mediated oral proficiency testing: considerations in
developing Simulated Oral Proficiency Interviews (SOPIs). In
Bolton, S.
, editor, TESTDAF: Grundlagen für die Entwicklung eines neuen
Sprachtests. Beiträge aus einem Expertenseminar [TESTDAF: foundations
of the development of a new language test].
Köln: VUB Gilde
, 87-106.
62.
Klein-Braley, C.1985:
A cloze-up on the C-test: a study in the construct validation of authentic tests
. Language Testing2,
76-104
.
63.
Klein-Braley, C. 1994: Language testing with the C-Test: a linguistic and
statistical investigation into the strategies used by C-Test takers, and the
prediction of C-Test difficulty. Unpublished Higher Thesis, Department of
Linguistics and Literature, University of Duisburg, Germany.
64.
Klein-Braley, C.1996a: Hunting unicorns: C-Tests, cloze tests and the RIP
(Removal of Information Procedure). In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
3.
Bochum: Brockmeyer
, 127-137.
65.
Klein-Braley, C.1996b: Towards a theory of C-Test processing. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
3.
Bochum: Brockmeyer
, 23-94.
66.
Klein-Braley, C.1997:
C-Tests in the context of reduced redundancy testing: an appraisal
. Language Testing14,
47-84
.
67.
Klein-Braley, C.
and
Raatz, U.1984:
A survey of research on the C-Test
. Language Testing1,
134-146
.
68.
Kline, R. B.1998: Principles and practice of structural equation modeling.
New York: Guilford
.
69.
Kolen, M. J.
and
Brennan, R. L.2004: Test equating, scaling, and linking: methods and
practices. 2nd edition.
New York: Springer
.
70.
Kontra, E. H.
and
Kormos, J.2006: Strategy use and the construct of C-tests. In
Grotjahn, R.
, editor, Der C-Test: Theorie, Empirie, Anwendungen/The C-test: theory,
empirical research, applications.
Frankfurt am Main: Peter Lang
, 121-138.
71.
Krekeler, C.2003:
Der kleine Unterschied - und keine Folgen? Grammatik in Sprachtests
für den Hochschulzugang [The role of grammar in language tests for
admission to higher education]
. Zeitschrift für Fremdsprachenforschung14,
107-147
.
72.
Kunnan, A. J.1998:
An introduction to structural equation modelling for language assessment research
. Language Testing15,
295-332
.
73.
Lee, Y.-W.2004:
Examining passage-related local item dependence (LID) and measurement
construct using Q3 statistics in an EFL reading comprehension test
. Language Testing21,
74-100
.
74.
Linacre, J. M.1989: Many-facet Rasch measurement.
Chicago: MESA Press
.
75.
Linacre, J. M.2002:
What do infit and outfit, mean-square and standardized mean?Rasch Measurement Transactions16,
878-878
.
76.
Linacre, J. M.2004a: A user’s guide to FACETS: Rasch-model computer
programs.
Chicago, IL
: Winsteps.com
77.
Linacre, J. M.2004b: A user’s guide to WINSTEPS-MINISTEP: Rasch-model
computer programs.
Chicago, IL
: Winsteps.com
78.
Linacre, J. M.
and
Wright, B. D.2002:
Construction of measures from many-facet data
. Journal of Applied Measurement3,
484-509
.
79.
Little, D.
and
Singleton, D.1992: The C-test as an elicitation instrument in second language
research. In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-test: theoretical foundations and practical applications]. Volume
1.
Bochum: Brockmeyer
, 173-192.
80.
McNamara, T. F.1996: Measuring second language performance.
London: Longman
.
81.
Messick, S.1989: Validity. In
Linn, R. L.
, editor, Educational measurement. 3rd edition.
New York: Macmillan
, 13-103.
82.
Messick, S.1995:
Validity of psychological assessment: validation of inferences from
persons’ responses and performances as scientific inquiry into
score meaning
. American Psychologist50,
741-749
.
83.
Myers, R.1990: Classical and modern regression with applications.
2nd edition.
Boston, MA: Duxbury
.
84.
Nassaji, H.2003:
Higher-level and lower-level text processing skills in advanced ESL reading comprehension
. The Modern Language Journal87,
261-276
.
85.
Negishi, M.1987:
The C-test: an integrative measure?IRLT Bulletin1,
3-26
.
86.
Oller, J. W., Jr.1983:A consensus for the eighties? In
Oller, J. W., Jr.
, editor, Issues in language testing research.
Rowley, MA: Newbury House
, 351-356.
87.
Purpura, J. E.1998:
Investigating the effects of strategy use and second language test
performance with high- and low-ability test takers: a structural equation
modelling approach
. Language Testing15,
333-379
.
88.
Raatz, U.
,
Grotjahn, R.
and
Wockenfuß, V.2006: Das TESTATT-Projekt: Entwicklung von C-Tests zur Evaluation
des Fremdsprachenstudiums [The TESTATT project: development of C-tests for the
evaluation of foreign language courses]. In
Grotjahn, R.
, editor, Der C-Test: Theorie, Empirie, Anwendungen [The C-test: theory,
empirical research, applications].
Frankfurt am Main: Peter Lang
, 85-99.
89.
Rasch, G.1960: Probabilistic models for some intelligence and attainment
tests. Copenhagen: Nielsen & Lydiche. Expanded edition, 1980,
Chicago, IL: University of Chicago Press
.
90.
Read, J.2000: Assessing vocabulary.
Cambridge: Cambridge University Press
.
91.
Rea-Dickins, P.1997: The testing of grammar in a second language. In
Clapham, C.
and
Corson, D.
, editors, Encyclopedia of language and education. Volume 7. Language
testing and assessment.
Dordrecht: Kluwer
, 87-97.
92.
Reckase, M. D.
,
Ackerman, T. A.
and
Carlson, J. E.1988:
Building a unidimensional test using multidimensional items
. Journal of Educational Measurement25,
193-203
.
93.
Schermelleh-Engel, K.
,
Moosbrugger, H.
and
Müller, H.2003:
Evaluating the fit of structural equation models: tests of significance and
descriptive goodness-of-fit measures
. Methods of Psychological Research Online8,
23-74
.
94.
Schumacker, R. E.
and
Lomax, R. G.2004: A beginner’s guide to structural equation
modeling. 2nd edition.
Mahwah, NJ: Lawrence Erlbaum
.
95.
Schwibbe, G.
and
Schwibbe, M.1987: Validierungsuntersuchungen zum Duisburg English Language
Test for Advanced Students (DELTA) [Validating the Duisburg English Language
Test for Advanced Students (DELTA)]. In
Grotjahn, R.
,
Klein-Braley, C.
and
Stevenson, D. K.
, editors, Taking their measure: the validity and validation of language
tests.
Bochum: Brockmeyer
, 255-274.
96.
Shohamy, E.1982:
Predicting speaking proficiency from Cloze tests: theoretical and practical
considerations for test substitution
. Applied Linguistics3,
161-171
.
97.
Sigott, G.2002: High-level processes in C-Test taking? In
Grotjahn, R.
, editor, Der C-Test: theoretische Grundlagen und praktische Anwendungen
[The C-Test: theoretical foundations and practical applications]. Volume
4.
Bochum: AKS-Verlag
, 67-82.
98.
Sigott, G.2004: Towards identifying the C-Test construct.
Frankfurt am Main: Peter Lang
.
99.
Sigott, G.2006: How fluid is the C-Test construct? In
Grotjahn, R.
, editor, Der C-Test: Theorie, Empirie, Anwendungen [The C-Test: theory,
empirical research, applications].
Frankfurt am Main: Peter Lang
, 139-146.
100.
Singleton, D.1999: Exploring the second language mental lexicon.
Cambridge: Cambridge University Press
.
101.
Singleton, D.
and
Singleton, E.2002: The C-Test and L2 acquisition/processing research.
In
Coleman, J. A.
,
Grotjahn, R.
and
Raatz, U.
, editors, University language testing and the C-Test.
Bochum: AKS-Verlag
, 143-168.
102.
Spolsky, B.2001: Closing the cloze. In
Pürschel, H.
and
Raatz, U.
, editors, Tests and translation: papers in memory of Christine
Klein-Braley.
Bochum: AKS-Verlag
, 1-20.
103.
Stemmer, B.1991: What’s on a C-Test taker’s mind: mental
processes in C-Test taking.
Bochum: Brockmeyer
.
104.
Stevens, J. P.2002: Applied multivariate statistics for the social sciences.
4th edition.
Mahwah, NJ: Lawrence Erlbaum
.
105.
Upshur, J. A.
and
Homburg, T. J.1983: Some relations among language tests at successive ability
levels. In
Oller, J. W., Jr.
, editor, Issues in language testing research.
Rowley, MA: Newbury House
, 188-202.
106.
Vollmer, H. J.1981: Why are we interested in ‘General Language
Proficiency’? In
Alderson, J. C.
and
Hughes, A.
, editors, Issues in language testing.
London: The British Council
, 152-171.
107.
Vollmer, H. J.
and
Sang, F.1983: Competing hypotheses about second language ability: a plea
for caution. In
Oller, J. W., Jr.
, editor, Issues in language testing research.
Rowley, MA: Newbury House
, 29-79.
108.
Weir, C. J.2005: Language testing and validation: an evidence-based
approach.
Basingstoke: Palgrave Macmillan
.
109.
Wright, B. D.
and
Masters, G. N.1982: Rating scale analysis.
Chicago: MESA Press
.
110.
Wright, B. D.
and
Masters, G. N.2002:
Number of person or item strata
. Rasch Measurement Transactions16,
888-888
.
111.
Wright, F. 1990: Testing foreign language ability: two approaches in theory
and practice. Unpublished M. Litt. Thesis, Trinity College Dublin, Ireland.
