Item response theory (IRT) methodology is introduced as a tool for improving assessment instruments used with people who have disabilities. Need for this approach in rehabilitation is emphasized; differences between IRT and classical test theory are clarified. Concepts essential to understanding IRT are defined, necessary data assumptions are identified, and specific data analysis techniques, using software such as TESTFACT and BILOG, are described. An example of IRT analysis applied to a subtest of the General Aptitude Test Battery (GATB) administered to people who have diverse disabilities is provided. Future potential uses of this approach in rehabilitation are outlined.
Get full access to this article
View all access options for this article.
References
1.
Aiken, L.R. (2003). Psychological testing and assessment (11th ed.). Boston: Allen & Bacon.
2.
Baker, F.B. (1985). The basics of item response theory. Portsmouth, NH: Heinemann.
Baker, F.B. (2001). The basics of item response theory (ERIC Clearing-house on Assessment and Evaluation, University of Maryland, College Park, MD). Retrieved October 30, 2004, from http://edres.org/irt/baker/
5.
Bock, R.D., Gibbons, R., Schilling, S.G., Muraki, R., Wilson, D.T., & Wood, R. (2003). Testfact 4. Lincolnwood, IL : Scientific Software International.
6.
Bolton, B. (2001). Handbook of measurement and evaluation in rehabilitation (3rd ed.). Gaithersburg, MD : Aspen.
7.
Borsboom, D., Mellenbergh, G.J., & van Heerden, J. (2003). The theoretical status of latent variables. Psychological Review, 110(2), 203— 219.
8.
Bullis, M., Reiman, J.W., Davis, C., & Reid, C. (1997). National field-testing of the “mini” version of the Transition Competence Battery for Adolescents and Young Adults Who Are Deaf. The Journal of Special Education, 31(3), 347—361.
9.
Bullis, M., Reiman, J., Reid, C., & Davis, C. (1995). Preliminary investigation of the “mini” version of the Transition Competence Battery for Deaf Adolescents and Young Adults. Assessment in Rehabilitation and Exceptionality, 2(3), 179—196.
10.
Cella, D., & Chang, C.H. (2000). A discussion of item response theory and its applications in health status assessment. Medical Care, 38(9, Suppl. 2), 1166—1172.
11.
Chang, C.H., & Gehlert, S. (2003). The Washington Psychosocial Seizure Inventory (WPSI): Psychometric evaluation and future applications. Seizure, 12(5), 261—267.
12.
Cohen, R.J., & Swerdlik, M.E. (2005). Psychological testing and assessment: An introduction to tests and measurement. Boston: McGraw-Hill.
13.
Cook, K.F., Monahan, P.O., & McHorney, C.A. (2003). Delicate balance between theory and practice: Health status assessment and item response theory. Medical Care, 41(5), 571—574.
14.
Du Toit, M. (Ed.). (2003). IRT from SSI: BILOG MG, MULTILOG, PARSCALE, TESTFACT. Lincolnwood, IL: Scientific Software International .
15.
Embretson, S.E., & Reise, S.P. (2000). Item response theory for psychologists. Mahwah, NJ: Erlbaum.
16.
Fayers, P.M., & Machin, D. (2000). Quality of life: Assessment, analysis, and interpretation. West Sussex, England: Wiley.
17.
Gumpel, T., & Wilson, M. (1996). Application of a Rasch analysis to the examination of the perception of facial affect among persons with mental retardation. Research in Developmental Disabilities, 17(2), 161—171.
18.
Haberman, S.J. (1977). Log-linear models and frequency tables with small expected cell counts. Annals of Statistics, 5, 1148—1169.
19.
Hahn, E.A., & Cella, D. (2003). Health outcomes assessment in vulnerable populations: Measurement challenges and recommendations. Archives of Physical Medicine and Rehabilitation, 84(Suppl. 2), S35—S42.
20.
Haley, S.M., Andres, P.L., Coster, W.J., Kosinski, M., Ni, P., & Jette, A.M. (2004). Short-form activity measure for post-acute care . Archives of Physical Medicine and Rehabilitation, 85(4), 649—660.
21.
Haley, S.M., Coster, W.J., Andres, P.L., Ludlow, L.H., Ni, P., Bond, T.L., et al. (2004). Activity outcome measurement for postacute care. Medical Care, 42(1, Suppl. 1), 149—161.
22.
Hambleton, R.K. (1989). Principles and selected applications of item response theory. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 147—200). New York: American Council on Education and Macmillan .
23.
Hambleton, R.K., & Swaminathan, H. (1985). Item response theory: Principles and applications. Boston: Kluwer Nijhoff.
24.
Hartigan, J. A., & Wigdor, A. K. (Eds.). (1989). Fairness in employment testing: Validity generalization, minority issues, and the General Aptitude Test Battery (Final report of the Committee on the General Aptitude Test Battery, Commission on Behavioral and Social Sciences and Education, National Research Council). Washington, DC: National Academy Press.
25.
Hays, R.D., Morales, L.S., & Reise, S.P. (2000). Item response theory and health outcomes measurement in the 21st century. Medical Care, 38(9, Suppl. 2), 1128—1142.
Jenkinson, C., Fitzpatrick, R., Garratt, A., Peto, V., & Stewart-Brown, S. (2001). Can item response theory reduce patient burden when measuring health status in neurological disorders? Results from Rasch analysis of the SF-36 physical functioning scale (PF-10) . Journal of Neurology, Neurosurgery, and Psychiatry, 71(2), 220—224.
28.
Jenkinson, C., Norquist, J.M., & Fitzpatrick, R. (2003). Deriving summary indices of health status from the Amyotrophic Lateral Sclerosis Assessment Questionnaires (ALSAQ-40 and ALSAQ-5). Journal of Neurology, Neurosurgery, and Psychiatry, 74(2), 242—245.
29.
Kopec, J.A., Esdaile, J.M., Abrahamowicz, M., Abenhaim, L., Wood-Dauphinee, S., Lamping, D.L., et al. (1996). The Quebec Back Pain Disability Scale: Conceptualization and development. Journal ofClinical Epidemiology , 49(2), 151—161.
30.
Lord, F.M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Erlbaum.
31.
McHorney, C.A. (2002). Use of item response theory to link 3 modules of functional status items from the Asset and Health Dynamics Among the Oldest Old study. Archives of Physical Medicine and Rehabilitation , 83(3), 383—394.
32.
McHorney, C.A., & Cohen, A.S. (2000). Equating health status measures with item response theory: Illustrations with functional status items. Medical Care, 38(9, Suppl. 2), 1143—1159.
33.
Mislevy, R.J., & Bock, R.D. (1990). BILOG 3: Item analysis and test scoring with binary logistic models (2nd ed.). Chicago : Scientific Software.
34.
Mungas, D., & Reed, B.R. (2000). Application of item response theory for development of a global functioning measure of dementia with linear measurement properties . Statistics in Medicine, 19(11—12), 1631—1644.
35.
Raju, N.S. (1990). Determining the significance of estimated signed and unsigned areas between two item response functions. Applied Psychological Measurement, 14, 197—207.
36.
Reckase, M.D. (1979). Unifactor latent trait models applied to multi-factor tests: Results and implications. Journal of Educational Statistics , 4, 207—230.
37.
Reid, C. (1993). Latent trait modeling of the General Aptitude Test Battery used with a rehabilitation client population: An investigation of model— data fit. Unpublished doctoral dissertation, Illinois Institute of Technology, Chicago.
38.
Reid, C. (1995). Application of item response theory to practical problems in assessment with people who have disabilities. Assessment in Rehabilitation and Exceptionality, 2(2), 89—96.
39.
Saliba, D., Orlando, M., Wenger, N.S., Hays, R.D., & Rubenstein, L.Z. (2000). Identifying a short functional disability screen for older persons. The Journals of Gerontology. Series A, Biological Sciences and MedicalSciences, 55(12), M750—M756.
40.
Santor, D.A., & Ransay, J.O. (1998). Progress in the technology of measurement: Applications of item response models. Psychological Assessment, 10(4), 345—359.
41.
Santor, D.A., Ramsay, J.O., & Zuroff, F.C. (1994). Nonparametric item analyses of the Beck Depression Inventory: Evaluating gender item bias and response option weights. Psychological Assessment, 6, 255—270.
42.
Stark, S., Chernyshenko, S., Chuah, D., Lee, W., & Wadlington, P. (2001). IRT Modeling Lab tutorial on item response theory. Retrieved October 30, 2004, from http://work.psych.uiuc.edu/irt/tutorial.asp
43.
Stoloff, M. L., & Couch, J. V. (Eds.). (1992). Computer use in psychology: A directory of software (3rd ed.). Washington, DC: American Psychological Association .
44.
Stout, W. (1987). A nonparametric approach for assessing latent trait unidimensionality. Psychometrika, 52, 589—617.
45.
U.S. Department of Labor. (1970). Manual for the General Aptitude Test Battery. Washington, DC: U.S. Government Printing Office.
46.
Van der Linden, W.J., & Hambleton, R.K. (1997). Handbook of modern item response theory. New York: Springer-Verlag .
47.
Ware, J.E., Jr., Bjorner, J.B., & Kosinski, M. (2000). Practical implications of item response theory and computerized adaptive testing: A brief summary of ongoing studies of widely used headache impact scales. Medical Care, 38(9, Suppl. 2), 1173—1182.
48.
Weiss, D.J. (1972). Review of the General Aptitude Test Battery. In O. K. Buros (Ed.), Seventh mental measurements yearbook (Vol. 2, pp. 1058—1061). Highland Park, NJ: Gryphon Press.
49.
Zimowski, M.F., Muraki, E., Mislevy, R.J., & Bock, R.D. (2003). BILOG-MG 3. Lincolnwood, IL : Scientific Software International.
