Despite theoretical differences between item response theory (IRT) and classical test theory (CTT), there is a lack of empirical knowledge about how, and to what extent, the IRT- and CTT-based item and person statistics behave differently. This study empirically examined the behaviors of the item and person statistics derived from these two measurement frameworks. The study focused on two issues: (a) What are the empirical relationships between IRT- and CTT-based item and person statistics? and (b) To what extent are the item statistics from IRT and those from CIT invariant across different participant samples? A large-scale statewide assessment database was used in the study. The findings indicate that the person and item statistics derived from the two measurement frameworks are quite comparable. The degree of invariance of item statistics across samples, usually considered as the theoretical superiority IRT models, also appeared to be similar for the two measurement fireworks.
Get full access to this article
View all access options for this article.
References
1.
Anastasi, A. (1988). Psychological testing (6th ed.). New York: Macmillan.
2.
Angoff, W. H. (1982). Use of difficulty and discrimination indices for detecting item bias. In R. A. Berk (Ed.), Handbook of methods for detecting test bias (pp. 96-116). Baltimore: Johns Hopkins University Press.
3.
Becker, D. F. , & Forsyth, R. A. (1992). An empirical investigation of Thurstone and IRT methods of scaling achievement tests. Journal of Educational Measurement, 29, 341-354.
4.
Clemans, W. V. (1993). Item response theory, vertical scaling, and something's awry in the state of test mark. Educational Assessment, 1, 329-347.
5.
Cole, N. S. , & Moss, P. A. (1993). Bias in test use. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 201-219). Phoenix, AZ: Oryx Press.
6.
Cook, L. L. , Eignor, D. R., & Taft, H. L. (1988). A comparative study of the effects of recency of instruction on the stability of IRT and conventional item parameter estimates. Journal of Educational Measurement, 25, 31-45.
7.
Crocker, L. , & Algina, J. (1986). Introduction to classical and modern test theory. New York: Holt, Rinehart & Winston.
8.
Donlon, G. (1984). The college board technical handbook for the Scholastic Aptitude Test and achievement tests. New York: College Entrance Examination Board.
9.
Englehard, G., Jr. (1990, April). Thorndike, Thurstone and Rasch: A comparison of their approaches to item-invariant measurement. Paper presented at the annual meeting of the American Educational Research Association, Boston. (ERIC Document Reproduction Services No. ED 320 921)
10.
Gall, M. D. , Borg, W. R., & Gall, J. P. (1996). Educational research: An introduction. White Plains, NY: Longman.
11.
Goldstein, H. , & Wood, R. (1989). Five decades of item response modeling. British Journal of Mathematical and Statistical Psychology, 42, 139-167.
12.
Hambleton, R. K. , & Jones, R. W. (1993). Comparison of classical test theory and item response theory and their applications to test development. Educational Measurement: Issues and Practice, 12(3), 38-47.
13.
Hambleton, R. K. , & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer.
14.
Hambleton, R. K. , Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
15.
Harris, D. J. (1991). A comparison of Angoff's Design I and Design II for vertical equating using traditional and IRT methodology. Journal of Educational Measurement, 28, 221-235.
16.
Hinkle, D. E. , Wiersma, W., & Jurs, S. G. (1988). Applied statistics for the behavioral sciences (2nd ed.). Boston: Houghton Mifflin.
17.
Kolen, M. J. (1981). Comparison of traditional and item response theory methods for equating tests. Journal of Educational Measurement, 18, 1-11.
18.
Lawson, S. (1991). One parameter latent trait measurement: Do the results justify the effort? In B. Thompson (Ed.),Advances in educational research: Substantive findings, methodological developments (Vol.1, pp. 159-168). Greenwich, CT: JAI.
19.
Lord, F. M. (1980).Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
20.
Lykken, D. T. (1968). Statistical significance of psychological research. Psychological Bulletin, 70, 155-159.
21.
McKinley, R. , & Mills, C. (1989). Item response theory: Advances in achievement and attitude measurement. In B. Thompson (Ed.), Advances in social science methodology (Vol.1, pp. 71-135). Greenwich, CT: JAI.
22.
Miller, M. D. , & Linn, R. L. (1988). Invariance of item characteristic functions with variations in instructional coverage. Journal of Educational Measurement, 25, 205-219.
23.
Mislevy, R. J. , & Bock, R. D. (1990). BILOG 3: Item analysis and test scoring with binary logistic models. Chicago: Scientific Software International.
24.
Peterson, N. S. , Cook, L. L., & Stocking, M. L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8, 137-156.
25.
Rudner, L. M. (1983). A closer look at latent trait parameter invariance. Educational and Psychological Measurement, 43, 951-955.
26.
Skaggs, G. , & Lissitz, R. W. (1986a). An exploration of the robustness of four test equating models. Applied Psychological Measurement, 10, 303-317.
27.
Skaggs, G. , & Lissitz, R. W. (1986b). IRT test equating: Relevant issues and a review of recent research. Review of Educational Research, 56, 495-529.
28.
Skaggs, G. , & Lissitz, R. W. (1988). Effect of examinee ability on test equating invariance. Applied Psychological Measurement, 12, 69-82.
29.
Thorndike, R. L. (1982). Educational measurement: Theory and practice. In D. Spearritt (Ed.), The improvement of measurement in education and psychology: Contributions of latent trait theory (pp. 3-13). Princeton, NJ: ERIC Clearinghouse of Tests, Measurements, and Evaluations. (ERIC Document Reproduction Service No. ED 222 545)
30.
Thurstone, L. L. (1947). The calibration of test items. American Psychologist, 2, 103-104.
