Despite the well-known theoretical advantages of item response theory (IRT) over classical test theory (CTT), research examining their empirical properties has failed to reveal consistent, demonstrable differences. Using Monte Carlo techniques with simulated test data, this study examined the behavior of item and person statistics obtained from these two measurement frameworks. The findings suggest IRT- and CTT-based item difficulty and person ability estimates were highly comparable, invariant, and accurate in the test conditions simulated. However, whereas item discrimination estimates based on IRT were accurate across most of the experimental conditions, CTT-based item discrimination estimates proved accurate under some conditions only. Implications of the results of this study for psychometric item analysis and item selection are discussed.
Get full access to this article
View all access options for this article.
References
1.
Baker, F. B. (1992). Item response theory: Parameter estimation techniques. New York: Marcel Dekker.
2.
Cook, L. L. , Eignor, D. R., & Taft, H. L. (1988). A comparative study of the effects of recency of instruction on the stability of IRT and conventional item parameter estimates. Journal of Educational Statistics, 25, 31-45.
3.
Crocker, L. , & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
4.
Fan, X. (1998). Item response theory and classical test theory: An empirical comparison of their item/person statistics. Educational and Psychological Measurement, 58, 357-381.
5.
Hambleton, R. K. , & Jones, R. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12, 38-47.
6.
Hambleton, R. K. , Jones, R. W., & Rogers, H. J. (1993). Influence of item parameter estimation errors in test development. Journal of Educational Measurement, 30, 143-155.
7.
Hambleton, R. K. , & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer.
8.
Hambleton, R. K. , Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
9.
Harwell, M. , Stone, C. A., Hsu, T. C., & Kirisci, L. (1996). Monte Carlo studies in item response theory. Applied Psychological Measurement, 20, 101-125.
10.
Lawson, S. (1991). One parameter latent trait measurement: Do the results justify the effort? In B. Thompson (Ed.), Advances in educational research: Substantive findings, methodological developments (Vol. 1, pp. 159-168). Greenwich, CT: JAI.
11.
Lord, F. M. (1953). The relation of test score to the trait underlying the test. Educational and Psychological Measurement, 13, 517-548.
12.
Maranon, P. P. , Garcia, M. I. B., & Costas, C. S. L. (1997). Identification of nonuniform differential item functioning: A comparison of Mantel-Haenszel and item response theory analysis procedures. Educational and Psychological Measurement, 57, 559-568.
13.
McKinley, R. , & Mills, C. (1989). Item response theory: Advances in achievement and attitude measurement. In B. Thompson (Ed.), Advances in social science methodology (Vol. 1, pp. 71-135). Greenwich, CT: JAI.
14.
Muraki, E. , & Bock, R. D. (1997). PARSCALE: IRT item analysis and test scoring for rating-scale data [Computer program]. Chicago: Scientific Software International.
15.
Ndalichako, J. L. , & Rogers, W. T. (1997). Comparison of finite state score theory, classical test theory, and item response theory in scoring multiple-choice items. Educational and Psychological Measurement, 57, 580-589.
16.
Nunnally, J. C. , & Berstein, I. (1994). Psychometric theory (3rd ed.). New York: McGraw-Hill.
17.
Park, D. G. , & Lautenshlager, G. J. (1990). Improving IRT item bias detection with iterative linking and ability scale purification. Applied Psychological Measurement, 14, 163-173.
18.
Rudner, L. M. (1983). A closer look at latent trait parameter invariance. Educational and Psychological Measurement, 43, 951-955.
19.
Thorndike, R. L. (1982). Educational measurement: Theory and practice. In D. Spearritt (Ed.), The improvement of measurement in education and psychology: Contributions of latent trait theory (pp. 3-13). Princeton, NJ: ERIC Clearinghouse of Tests, Measurements, and Evaluations. (ERIC Document Reproduction Service No. ED 222 545)
20.
Tinsley, H. E. , & Dawis, R. V. (1977). Test-free person measurement with the Rasch simple logistic model. Applied Psychological Measurement, 1, 483-487.
21.
van de Linden, W. J. , & Hambleton, R. K. (1997). Item response theory: Brief history, common models, and extensions. In W. J. van de Linden & R. K. Hambleton (Eds.), Handbook of modern item response theory (pp. 1-28). New York: Springer.
22.
Veerkamp, W. J. J. , & Berger, M. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22, 203-226.
