This study asks, how well does an item response theory (IRT) based computerized adaptive NEO PI-R work? To explore this question, real-data simulations (N = 1,059) were used to evaluate a maximum information item selection computerized adaptive test (CAT) algorithm. Findings indicated satisfactory recovery of full-scale facet scores with the administration of around four items per facet scale. Thus, the NEO PI-R could be reduced in half with little loss in precision by CAT administration. However, results also indicated that the CAT algorithm was not necessary. We found that for many scales, administering the "best" four items per facet scale would have produced similar results. In the conclusion, we discuss the future of computerized personality assessment and describe the role IRT methods might play in such assessments.
American College Testing
. (1993). COMPASS user's guide. Iowa City, IA: Author.
2.
Ben-Porath, Y. S.
, & Butcher, J. N. (1986). Computers in personality assessment: A brief past, an ebullient present, and an expanding future. Computers in Human Behavior, 2, 167-182.
3.
Biskin, B. H.
, & Kolotkin, R. L. (1977). Effects of computerized administration of scores on the Minnesota Multiphasic Personality Inventory. Applied Psychological Measurement, 1, 543-549.
4.
Bock, R. D.
, & Mislevy, R. J. (1982). Adaptive EAP estimation of ability in a microcomputer environment. Applied Psychological Measurement, 6, 431-444.
5.
Costa, P. T., Jr.
, & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO PI-R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources.
6.
Costa, P. T., Jr.
, & McCrae, R. R. (1994). Bibliography for the revised NEO Personality Inventory (NEO P1-R) and NEO Five-Factor Inventory (NEO-FFI). Odessa, FL: Psychological Assessment Resources.
7.
Daniel, M. H.
(1999). Behind the scenes: Using new measurement methods on the DAS and KAIT. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every psychologist should know (pp. 37 -63). Mahwah, NJ: Erlbaum.
8.
Divgi, D. R.
(1986). Determining the sensitivity of CAT-ASVAB scores to changes in item response cnrves with medium of administration (Report No. 86-189). Alexandria, VA: Center for Naval Analyses.
9.
Divgi, D. R.
, & Stoloff, P. H. (1986). Effect of the medium of administration on AS`VAB item response curves (Report No. 86-24). Alexandria, VA: Center for Naval Analyses.
10.
Dodd, B. G.
, de Ayala, R. J., & Koch, W. R. (1995). Computerized adaptive testing with polytomous items. Applied Psychological Measurement, 19, 5-22.
11.
Drasgow, F.
, & Olson-Buchanan, J. B. (1999). Innovations in computerized assessment. Hillsdale, NJ: Erlbaum.
12.
Embretson, S. E.
, & Reise, S. P. (2000). Item Response Theory for psychologists. Mahwah, NJ: Erlbaum.
13.
Fekken, G. C.
, & Holden, R. R. (1992). Response latency evidence for viewing personality traits as schema indicators. Journal of Research in Personality, 26, 103-120.
14.
Fekken, G. C.
, & Holden, R. R. (1994). The construct validity of differential response latencies in structured personality tests. Canadian Journal of Behavioral Science, 26, 104-120.
15.
Fitzpatrick, S. J.
, Choi, S. W., Chen, S., Hou, L., & Dodd, B. G. (1994). IRTINFO: A SAS macro program to compute item and test information. Applied Psychological Measurement, 18, 390-390.
16.
Green, B. F.
, Bock, R. D., Humphreys, L. G., Linn, R. L., & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21, 347-360.
17.
Hambleton, R. K.
, & Swaminathan, H. (1985). Item response theory: Principles and applications. Norwell, MA: Kluwer Academic.
18.
Hattie, J.
(1984). An empirical study of various indices for determining unidimensionality. Multivariate Behavioral Research, 19, 49-78.
19.
Hattie, J.
(1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139-164.
20.
Hetter, R. D.
, Segall, D. O., & Bloxom, B. M. (1994). A comparison of item calibration media in computerized adaptive testing. Applied Psychological Measurement, 18, 197-204.
21.
Holden, R. R.
, & Fekken, G. C. (1993). Can personality test response latencies have construct validity? Issues of reliability and convergent and discriminant validity. Personality and Individual Differences, 15, 243-248.
22.
Lee, J. A.
, Moreno, K. E., & Sympson, J. B. (1986). The effects of mode of test administration on test performance. Educational and Psychological Measurement, 46, 467-474.
23.
McDonald, R. P.
(1981). The dimensionality of tests and items. British Journal of Mathematical and Statistical Psychology, 34, 100-117.
24.
McDonald, R. P.
(1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.
25.
Mead, A. D.
, & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: A meta-analysis. Psychological Bulletin, 114, 449-458.
26.
Meijer, R. R.
, & Nering, M. L. (1999). Computerized adaptive testing: Overview and introduction. Applied Psychological Measurement, 23, 187-194.
27.
Mills, C. N.
(1999). Development and introduction of computer adaptive graduate record examinations general test. In F. Drasgow & J. B. Olson-Buchanan (Eds.), Innovations in computerized assessment (pp. 117-135). Mahwah, NJ: Erlbaum.
28.
Millsap, R. E.
, & Everson, H. T. (1993). Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement, 17, 297-334.
29.
Muraki, E.
, & Bock, R. D. (1997). PARSCALE: IRT item analysis and test scoring for rating-scale data. Chicago: Scientific Software International.
30.
Neuman, G.
, & Baydoun, R. (1998). Computerization of paper-and-pencil test: When are they equivalent?Applied Psychological Measurement, 22, 71-83.
31.
Piedmont, R. L.
, McCrae, R. R., Riemann, R., & Angleitner, A. (2000). On the invalidity of validity scales: Evidence from self-reports and observer ratings in volunteer samples. Journal of Personality and Social Psychology, 78, 582-593.
32.
Reise, S. P.
(1999). Personality measurement issues viewed through the eyes of IRT. In S. E. Embretson & S. L. Hershberger (Eds.), The new rules of measurement: What every psychologist should know (pp. 219-241). Mahwah, NJ: Erlbaum.
33.
Reise, S. P.
, & Henson, J. M. (2000). Fitting polytomous IRT models to the NEO PI-R facet scales. Manuscript submitted for publication.
34.
Reise, S. P.
, & Waller, N. G. (1990). Fitting the two-parameter model to personality data. Applied Psychological Measurement, 14, 45-58.
35.
Samejima, F.
(1969). Estimation of latent ability using a response pattern of graded scores. Psychometrika Monograph, No. 17.
36.
Santor, D. A.
, & Ramsay, J. O. (1998). Progress in the technology of measurement: Applications of item response models. Psychological Assessment, 10, 345-359.
37.
Spray, J. A.
, Ackerman, T. A., Reckase, M. D., & Carlson, J. E. (1989). Effect of the medium of item presentation on examinee performance and item characteristics. Journal of Educational Measurement, 26, 261-271.
38.
Steinberg, L.
(1994). Context and serial order effects in personality measurement: Limits on the generality of "measuring changes the measure."Journal of Personality and Social Psychology, 66, 341-349.
39.
Steinberg, L.
, & Thissen, D. (1995). Item response theory in personality research. In P. E. Shrout & S. T. Fiske (Eds.) Personality research, methods, and theory: A Festschrift honoring Donald W. Fiske (pp. 161-182). Hillsdale, NJ: Erlbaum.
40.
Sykes, R. C.
, & Ito, K. (1997). The effects of computer administration on scores and item parameter estimates of an IRT-based licensure examination. Applied Psychological Measurement, 21, 57-63.
41.
Sukigara, M.
(1996). Equivalence between computer and booklet administrations of the new Japanese version of the MMPI. Educational and Psychological Measurement, 27, 361-370.
42.
Wainer, H.
, Dorans, N. J., Faugher, R., Green, B. F., Mislevy, R. J., Steinberg, L., & Thissen, D. (1990). Computerized adaptive testing: A primer. Hillsdale, NJ: Erlbaum.
43.
Waller, N. G.
, & Reise, S. P. (1989). Computerized adaptive personality assessment: An illustration with the absorption scale. Journal of Personality and Social Psychology, 57, 1051-1058.
44.
Watson, C. G.
, Thomas, D., & Anderson, P. E. D. (1992). Do computer-administered Minnesota Multiphasic Personality Inventories underestimate booklet-based scores?Journal of Clinical Psychology, 48, 744-748.
45.
Weiss, D. J.
(1982). Improving measurement quality and efficiency with adaptive testing. Applied Psychological Measurement, 4, 273-285.
46.
Weiss, D. J.
(1985). New horizons in testing: Latent trait test theory and computerized adaptive testing. New York: Academic Press.
