This study is about relations between classical test theory (CTT) and item response theory (IRT). It is shown that CTT is based on the assumption that measures are exchangeable, whereas IRT is based on conditional independence. Thus, IRT is presented as an extension of CTT, and concepts from both theories are related to one another. Furthermore, it is demonstrated that IRT can be used to provide CTT statistics in situations where CTT fails. Reliability, for instance, can be determined even though a test was not administered to the intended population.
Get full access to this article
View all access options for this article.
References
1.
Bechger, T. M., Maris, G., Béguin, A., & Verstralen, H. H. F. M. (2003). Combining classical test theory and item response theory. R&D Report 2003-4, Cito, Arnhem, The Netherlands.
2.
Bechger, T. M., Verstralen, H. H. F. M., & Verhelst, N. D. (2002). Equivalent linear logistic test models. Psychometrika, 67, 123-136.
3.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 395-479). Reading, MA: Addison-Wesley.
4.
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37-46.
5.
Cohen, J. (1968). Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin, 70, 213-220.
6.
Davis, P. J., & Rabinowitz, P. (1984). Methods of numerical integration (2nd ed.). New York: Academic Press.
7.
Ebel, R. L., & Frisbie, D. A. (1986). Essentials of educational measurement. Englewood Cliffs, NJ: Prentice Hall.
8.
Fischer, G. H. (1995). The linear logistic test model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications. New York: Springer.
9.
Glas, C. A. W., & Verhelst, N. D. (1995). Tests of fit for polytomous Rasch models. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications. New York: Springer.
10.
Gustafsson, J. E. (1977). The Rasch model for dichotomous items: Theory, applications and a computer program. Rep. no. 85, Institute of Education, University of Göteborg.
11.
Kolen, M. J., Zeng, L., & Hanson, B. A. (1996). Conditional standard errors of measurement for scale scores using IRT. Journal of Educational Measurement, 33, 129-140.
12.
Lee, W.-C., Hanson, B. A., & Brennan, R. L. (2002). Estimating consistency and accuracy indices for multiple classifications. Applied Psychological Measurement, 26, 412-432.
13.
Livingstone, S. A., & Lewis, C. (1995). Estimating the consistency and accuracy of classifications based on test scores. Journal of Educational Measurement, 32, 179-197.
14.
Loevinger, J. (1948). The technique of homogeneous tests compared with some aspects of scale analysis and factor analysis. Psychological Bulletin, 45, 507-530.
15.
Lord, F. M. (1953). The relation of the reliability of multiple-choice tests to the distribution of item difficulties. Psychometrika, 17, 181-194.
16.
Lord, F. M. (1983). Applications of item response theory to practical testing problems. Englewood Cliffs, NJ: Lawrence Erlbaum.
17.
Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. London: Addison-Wesley.
18.
Lord, F. M., & Wingersky, M. S. (1984). Comparison of IRT true-score and equipercentile observed-score “equatings.” Applied Psychological Measurement, 8, 453-461.
19.
Mellenbergh, G. J. (1994). A unidimensional latent trait model for continuous item responses. Multivariate Behavioral Research, 29(3), 223-236.
20.
Mellenbergh, G. J. (1996). Measurement precision in test score and item response models. Psychological Methods, 1(3), 293-299.
21.
Mokken, R. J. (1971). A theory and procedure of scale analysis: With applications in political research. The Hague, The Netherlands: Mouton.
22.
Muraki, E. (1992). A generalized partial credit model: Application of an EM-algorithm. Applied Psychological Measurement, 16, 159-176.
23.
Muraki, E. (1993). Information functions of the generalized partial credit model. Applied Psychological Measurement, 17, 351-363.
24.
Nicewander, W. A. (1993). Some relationships between the information function of IRT and the signal/noise ratio and reliability coefficient of classical test theory. Psychometrika, 58, 139-141.
25.
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests. Kopenhagen, Denmark: Nissen and Lydicke.
26.
Rost, J. (1996). Lehrbuch Testtheorie, Testkonstruktion [Textbook for test theory and test construction]. Bern, Switzerland: Hans Huber.
27.
Samejima, F. (1994). Estimation of reliability coefficients using the test information function and its modifications. Applied Psychological Measurement, 18, 229-244.
28.
Sluijter, C. (1998).Toetsen en beslissen. Toetsing bij doorstroombeslissingen in het voorgezet onderwijs [Tests and decision making: Making placement decisions in secondary education]. Unpublished doctoral dissertation, Cito, Arnhem, The Netherlands.
29.
Spray, J. A., & Reckase, M. D. (1994). The selection of test items for decision making with a computer adaptive test. Paper presented at the national meeting of the National Council on Measurement in Education, New Orleans, LA.
30.
Steyer, R., & Eid, M. (1993). Messen und testen [Measuring and testing]. Berlin: Springer-Verlag.
31.
Suppes, P., & Zanotti, M. (1981). When are probabilistic explanations possible? Synthese, 48, 191-199.
32.
Thissen, D. (1990). Reliability and measurement precision. In H. Wainer, N. J. Dorans, R. Flaugher, B. F. Green, R. L. Mislevy, L. Steinberg, & D. Thissen (Eds.), Computerized adaptive testing: A primer (pp. 161-186). Hillsdale, NJ: Lawrence Erlbaum.
33.
Thissen, D., Pommerich, M., Billeaud, K., & Williams, V. S. L. (1995). Item response theory for scores on tests including polytomous items with ordered responses. Applied Psychological Measurement, 19, 39-49.
34.
Verhelst, N. D., & Glas, C. A. W. (1995). The one parameter logistic model. In G. H. Fischer & I. W. Molenaar (Eds.), Rasch models: Foundations, recent developments, and applications. New York: Springer.
35.
Verhelst, N. D., Glas, C. A. W., & Verstralen, H. H. F. M. (1995). One parameter logistic model OPLM (Computer software manual). Arnhem, The Netherlands: Cito.
36.
Verstralen, H. H. F. M. (1997a). A logistic latent class model for multiple choice items. R&D report, Cito, Arnhem, The Netherlands.
37.
Verstralen, H. H. F. M. (1997b). OPTAL: Inverse OPLAT and item and test characteristics in populations. Arnhem, The Netherlands: Cito.
38.
von Davier, M. (1994). WINMIRA: A Windows 3.x program for analysis with the Rasch model, with the latent class model, and with the mixed Rasch model. Kiel, Germany: Institute for Science Education (IPN).
