Item response theory (IRT) seeks to model the way in which latent psychological constructs manifest themselves in terms of observable item responses; this information is useful when developing, evaluating, and scoring tests. After providing an overview of the most popular IRT models (i.e., those applicable to dichotomously keyed items) and contrasting them with the techniques used in classical test theory (CTT), the authors illustrate the application of IRT using data from the recently revised Myers-Briggs Type Indicator. These results highlight a number of IRT’s advantages, including (a) detailed descriptions of the performance of individual items, (b) indices of item- and scale-level precision that are free to vary across the full range of possible scores, (c) assessments of item- and test-level bias with respect to demographic subgroups, (d) measures of response-profile quality, and (e) computer-adaptive testing, which can dramatically reduce testing time.
Get full access to this article
View all access options for this article.
References
1.
Bock, R. D. (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories. Psychometrika, 37, 29-51.
2.
Bock, R. D. , & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443-445.
3.
Bock, R. D. , & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
4.
Brown, R. D. , & Harvey, R. J. (1998, April). Computer-adaptive testing and test-retest reliability in a “Big-Five” personality inventory. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Dallas.
5.
Drasgow, F. (1987). A study of the measurement bias of two standardized psychological tests. Journal of Applied Psychology, 72, 19-29.
6.
Drasgow, F. , & Hulin, C. L. (1990). Item response theory. In M. D. Dunnette & L. M. Hough (Eds.), Handbook of industrial & organizational psychology (2nd. ed., Vol. 1, pp. 577-636). Palo Alto, CA: Consulting Psychologists Press.
7.
Drasgow, F. , Levine, M. V., & McLaughlin, M. E. (1987). Detecting inappropriate test scores with optimal and practical appropriateness indices. Applied Psychological Measurement, 11, 59-79.
8.
Drasgow, F. , & Parsons, C. K. (1983). Application of unidimensional item response theory models to multidimensional data. Applied Psychological Measurement, 7, 189-199.
9.
Embretson, S. E. (1996). The new rules of measurement. Psychological Assessment, 8(4), 341-349.
10.
Gierl, M. J. , & Ackerman, T. (1996). Software review: XCALIBRE—Marginal maximum-likelihood estimation program, Windows version 1.10. Applied Psychological Measurement, 20, 303-307.
11.
Graff, R. W. , Larrimore, M., Whitehead, G. I., & Hopson, N. W. (1991). Career counseling practices: A survey of college/university counseling centers. Poster presented at the annual meeting of the American Psychological Association, San Francisco.
12.
Gratias, M. , & Harvey, R. J. (1998, April). Gender and ethnicity-based differential item functioning in the Myers-Briggs Type Indicator. Paper presented at the Annual Conference of the Society for Industrial and Organizational Psychology, Dallas.
13.
Hambleton, R. K. , Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.
14.
Harvey, R. J. (1997, April). Computer adaptive testing, differential item functioning, faking, and the MBTI. In R. J. Harvey (Chair), Using item response theory to address assessment challenges in I/O. Symposium conducted at the Annual Conference of the Society for Industrial and Organizational Psychology, St. Louis.
15.
Harvey, R. J. , & Murry, W. D. (1994). Scoring the Myers-Briggs Type Indicator: Empirical comparison of preference score versus latent-trait methods. Journal of Personality Assessment, 62, 116-129.
16.
Harvey, R. J. , Murry, W. D., & Stamoulis, D. (1995). Unresolved issues in the dimensionality of the Myers-Briggs Type Indicator. Educational and Psychological Measurement, 55, 535-544.
17.
Harvey, R. J. , & Thomas, L. (1996). Using item response theory to score the Myers-Briggs Type Indicator: Rationale and research findings. Journal of Psychological Type, 37, 16-60.
18.
Hulin, C. L. , Drasgow, F., & Parsons, C. K. (1983). Item response theory: Application to psychological measurement. Homewood, IL: Dow Jones-Irwin.
19.
Johnson, D. A. , & Saunders, D. R. (1990). Confirmatory factor analysis of the Myers-Briggs Type Indicator Expanded Analysis Report. Educational and Psychological Measurement, 50, 561-571.
20.
Laatsch, L. , & Choca, J. (1994). Cluster-branching methodology for adaptive testing of the adaptive category test. Psychological Assessment, 6, 1-7.
21.
Lord, F. M. (1952). A theory of test scores (Psychometric Monograph No. 7).
22.
Lord, F. M. (1968). An analysis of the Verbal Scholastic Aptitude Test using Birnbaum’s three-parameter logistic model. Educational and Psychological Measurement, 28, 989-1020.
23.
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum.
24.
Lord, F. M. , & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.
25.
Mislevy, R. J. , & Bock, R. D. (1983). BILOG: Maximum likelihood item analysis and test scoring-logistic models. Chicago: Scientific Software International.
26.
Muthen, B. (1984). A general structural equation model with dichotomous, ordered categorical, and continuous latent variable indicators. Psychometrika, 49, 115-132.
27.
Myers, I. B. , & McCaulley, M. H. (1985). MBTI manual: A guide to the development and use of the Myers-Briggs Type Indicator. Palo Alto, CA: Consulting Psychologists Press.
28.
Myers, I. B. , McCaulley, M. H., Quenk, N., & Hammer, A. L. (1998). MBTI manual: A guide to the development and use of the Myers-Briggs Type Indicator (Rev. ed.). Palo Alto, CA: Consulting Psychologists Press.
29.
Samejima, F. (1979). A new family of models for the multiple-choice item (Research Report 79-4 prepared under Office of Naval Research contract N00014-77-C-360, NR 150-402). Knoxville: Department of Psychology, University of Tennessee.
30.
Sands, W. A. , Waters, B. K., & McBride, J. R. (1997). Computertized adaptive testing: From inquiry to operation. Washington, DC: American Psychology Association.
31.
Thissen, D. , & Steinberg, L. (1985). A response model for multiple choice items. Psychometrika, 49, 501-519.
32.
Tucker, L. R. (1946). Maximum validity of a test with equivalent items. Psychometrika, 11, 1-13.
33.
van der Linden, W. , & Hambleton, R. K. (1997). Handbook of modern item response theory. Heidelberg, Germany: Springer-Verlag.
34.
Waller, N. G. , & Reise, S. P. (1989). Computerized adaptive personality assessment: An illustration with the Absorption Scale. Journal of Personality and Social Psychology, 37, 1051-1058.
35.
Wright, B. D. (1977). Solving measurement problems with the Rasch model. Journal of Educational Measurement, 14, 97-116.
