Placement and screening tests serve important functions, not only with regard to placing learners at appropriate levels of language courses but also with a view to maximizing the effectiveness of administering test batteries. We examined two widely reported formats suitable for these purposes, the discrete decontextualized Yes/No vocabulary test and the embedded contextualized C-test format, in order to determine which format can explain more variance in measures of listening and reading comprehension. Our data stem from a large-scale assessment with over 3000 students in the German secondary educational context; the four measures relevant to our study were administered to a subsample of 559 students. Using regression analysis on observed scores and SEM on a latent level, we found that the C-test outperforms the Yes/No format in both methodological approaches. The contextualized nature of the C-test seems to be able to explain large amounts of variance in measures of receptive language skills. The C-test, being a reliable, economical and robust measure, appears to be an ideal candidate for placement and screening purposes. In a side-line of our study, we also explored different scoring approaches for the Yes–No format. We found that using the hit rate and the false-alarm rate as two separate indicators yielded the most reliable results. These indicators can be interpreted as measures for vocabulary breadth and as guessing factors respectively, and they allow controlling for guessing.
AldersonJ. C. (2005). The vocabulary size placement test. In Diagnosing foreign language proficiency (pp. 79–96). London: Continuum.
2.
AldersonJ. C.HuhtaA. (2005). The development of a suite of computer-based diagnostic tests based on the Common European Framework. Language Testing, 22(3), 301–320.
3.
AldersonJ. C.KremmelB. (2013). Re-examining the content validation of a grammar test: The (im)possibility of distinguishing vocabulary and structural knowledge. Language Testing, 30(4), 535–556.
4.
AsparouhovT.MuthénB. (2005). Multivariate statistical modeling with survey data. Pro-ceedings of the Federal Committee on Statistical Methodology (FCSM) Research Conference.
5.
Author et al. (2007). [to come]
6.
Author et al. (2008). [to come]
7.
BachmanL. F.PalmerA. (1996). Language testing in practice. Oxford: Oxford University Press.
8.
BeeckmansR.EyckmansJ.JanssensV.DufranneM.Van de VeldeH. (2001). Examining the Yes/No test: Some methodological issues in theory and practice. Language Testing, 18(3), 235–274.
9.
ChapelleC. A. (1994). Are C-tests valid measures for L2 vocabulary research?Second Language Research, 10(2), 157–187.
10.
ColemanJ. A. (1996). Studying languages: A survey of British and European students: The proficiency, background, attitudes and motivations of students of foreign languages in the United Kingdom and Europe. (pp. 136–161). London: CILT.
11.
Council of Europe. (2001). A Common European Framework of Reference for Language Learning and Teaching. Cambridge: Cambridge University Press.
12.
EckesT. (2014). Die onDaF-TestDaF-Vergleichsstudie: Wie gut sagen Ergebnisse im onDaF Erfolg oder Misserfolg beim TestDaF vorher? In GrotjahnR. (Ed.) Der C-test: Aktuelle Tendenzen. [The C-test: Current trends] (pp. 137–162). Frankfurt/Main: Lang.
13.
EckesT.GrotjahnR. (2006). A closer look at the construct validity of C-tests. Language Testing, 23(3), 290–325.
14.
ElderC.von RandowJ. (2008). Exploring the utility of a Web-based English language screening tool. Language Assessment Quarterly, 5(3), 173–194.
15.
FreyA.SeitzN. N. (2009). Multidimensional adaptive testing in educational and psychological measurement: Current state and future challenges. Studies in Educational Evaluation, 35(2), 89–94.
16.
GrotjahnR.(2002). Konstruktion und Einsatz von C-Tests: Ein Leitfaden für die Praxis [Construction and use of C-test: a guide for practical implementation]. In GrotjahnR. (Ed.), Der C-Test. Theoretische Grundlagen und praktische Anwendungen [The C-test. Theoretical foundations and practical applications]. Bochum: AKS, 211–225.
17.
HarringtonM.CareyM. (2009). The online Yes/No test as a placement tool. System, 37(4), 614−626.
18.
HarschC.HartigJ. (2010). Empirische und inhaltliche Analyse lokaler Abhängigkeiten im C-Test. In: GrotjahnR. (Ed.), Der C-Test: Beiträge aus der aktuellen Forschung/The C-Test: Contributions from current research (pp. 193–204). Frankfurt/Main: Lang.
19.
HarschC.PantH. A.KöllerO. (Eds.). (2010). Calibrating standards-based assessment tasks for English as a first foreign language. Standard-setting procedures in Germany. Münster: Waxmann.
HarschC.SchröderK. (2008). Schülerkompetenzen im Englischen: Textrekonstruktion: C-Test. In: DESI-Konsortium (Ed.), Ergebnisse der DESI-Studie (pp. 149–156). Weinheim: Beltz.
22.
HastingsA. J. (2002). Error analysis of an English C-test: Evidence for integrated processing. In GrotjahnR. (Ed.), Der C-Test. Theoretische Grundlagen und praktische Anwendungen [The C-test. Theoretical foundations and practical applications]. Bochum: AKS, 53–66.
23.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
24.
HuibregtseI.AdmiraalW.MearaP. (2002). Scores on a Yes–No vocabulary test: Correction for guessing and response style. Language Testing, 19, 227–245.
25.
KarimiN. (2011). C-test and vocabulary knowledge. Language Testing in Asia, 1, 7–38.
26.
KieferT.RobitzschA.WuM. (2014). TAM: Test Analysis Modules. R package version 1.0–2. Retrieved May9, 2014, from http://CRAN.R-project.org/package=TAM
27.
Klein-BraleyC. (1985). A cloze-up on the C-test. Language Testing, 2, 76–104.
28.
Klein-BraleyC. (1997). C-tests in the context of reduced redundancy testing: An appraisal. Language Testing, 14, 47–84.
29.
Klein-BraleyC.RaatzU. (1984). A survey of research on the C-test. Language Testing, 1, 134–146.
30.
KöllerO.KniggeM.TeschB. (Eds.) (2010). Sprachliche Kompetenzen im Ländervergleich [Language proficiency in Germany: A comparison study]. Münster: Waxmann.
31.
Kultusministerkonferenz der Länder (KMK). (2003). Bildungsstandards für die erste Fremdsprache (Englisch/Französisch) für den Mittleren Abschluss [Educational Standards for the first foreign language English for the middle secondary school]. Darmstadt: Luchterhand.
32.
KunnanA. J. (1998). An introduction to structural equation modelling for language assessment research. Language Testing, 15, 295–332.
33.
LauferB.GoldsteinZ. (2004). Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language Learning, 54(3), 399–436.
34.
LauferB.NationP. (1999). A vocabulary size test of controlled productive ability. Language Testing, 16(1), 33–51.
35.
MacCallumR. C.BrowneM. W.SugawaraH. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1, 130–149.
36.
MearaP. M. (2005a). X_Lex: the Swansea Vocabulary Levels Test. v2.05. Swansea: Lognostics.
37.
MearaP. M. (2005b). Lex vocabulary tests v2.0. Swansea: University of Wales, Centre for Applied Language Studies.
NationP. (2006). How large a vocabulary is needed for reading and listening?The Canadian Modern Language Review, 63(1), 59–81.
45.
Pellicer-SánchezA.SchmittN. (2012). Scoring Yes–No vocabulary tests: Reaction time vs. nonword approaches. Language Testing, 29(4), 489–509.
46.
PurpuraJ. E. (1998). Investigating the effects of strategy use and second language test performance with high- and low-ability test takers: A structural equation modelling approach. Language Testing, 15, 333–379.
47.
QianD. D. (2008). From single words to passages: Contextual effects on predictive power of vocabulary measures for assessing reading performance. Language Assessment Quarterly, 5(1), 1–19.
48.
QianD. D.SchedlM. (2004). Evaluation of an in-depth vocabulary knowledge measure for assessing reading performance. Language Testing, 21, 28–52.
49.
ReadJ. (2000). Assessing vocabulary. Cambridge: Cambridge University Press.
50.
ReadJ. (2013). Research timeline: Second language vocabulary assessment. Language Teaching, 46(1), 41–52.
51.
ReadJ.ChapelleC. A. (2001). A framework for second language vocabulary assessment. Language Testing, 18(1), 1–32.
52.
ReichertM.BrunnerM.MartinR. (2014). Do test takers with different language backgrounds take the same C-test? The effect of native language on the validity of C-tests. In GrotjahnR. (Ed.), Der C-test: Aktuelle Tendenzen. The C-test: Current trends (pp. 109–136). Frankfurt/Main: Lang.
53.
RuppA. A.VockM.HarschC.KöllerO. (2008): Developing standards-based assessment tasks for English as a first foreign language – context, processes and outcomes in Germany. Münster: Waxmann.
54.
ShiotusT.WeirC. (2007). The relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance. Language Testing, 24(1), 99–128.
55.
SigottG. (2004). Towards identifying the C-test construct. Frankfurt/Main: Lang.
56.
SigottG. (2006). How fluid is the C-Test construct? In GrotjahnR. (Ed.), Der C-Test: Theorie, Empirie, Anwendungen [The C-Test: Theory, empirical research, applications] (pp. 139–46). Frankfurt/Main: Lang.
57.
StemmerB. (1991). What’s on a C-test taker’s mind? Mental processes in C-test taking. Bochum: Brockmeyer.
58.
WarmT. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427–450.
59.
YanD.von DavierA. A.LewisC. (2014). Computerized multistage testing: Theory and applications. Boca Raton, FL: Chapman & Hall/CRC.
