Traditional methods for item selection in computerized adaptive testing only focus on item information without taking into consideration the time required to answer an item. As a result, some examinees may receive a set of items that take a very long time to finish, and information is not accrued as efficiently as possible. The authors propose two item-selection criteria that utilize information from a lognormal model for response times. The first modifies the maximum information criterion to maximize information per time unit. The second is an inverse time-weighted version of a-stratification that takes advantage of the response time model, but achieves more balanced item exposure than the information-based techniques. Simulations are conducted to compare these procedures against their counterparts that ignore response times, and efficiency of estimation, time-required, and item exposure rates are assessed.
ChenS. Y.AnkenmannR. D. (2004). Effects of practical constraints on item selection rules at the early stages of computerized adaptive testing. Journal of Educational Measurement, 41, 149–174.
6.
ChenS. Y.AnkenmannR. D.ChangH. H. (2000). A comparison of item selection rules at the early stages of computerized adaptive testing. Applied Psychological Measurement, 24, 241–255.
7.
EmbretsonS. E. (1998). A cognitive design system approach to generating valid tests: Application to abstract reasoning. Psychological Methods, 3, 300–396.
8.
GorinJ. S. (2005). Manipulation of processing difficulty on reading comprehension test questions: The feasibility of verbal item generation. Journal of Educational Measurement, 42, 351–373.
9.
HauK. T.ChangH. H. (2001). Item selection in computerized adaptive testing: Should more discriminating items be used first?Journal of Educational Measurement, 38, 249–266.
10.
Klein EntinkR. H.FoxJ.-P.FoxJ.-P.van der LindenW. J. (2009). A Box-Cox normal model for response times. British Journal of Mathematical and Statistical Psychology, 62, 621–640.
11.
MarisE. (1993). Additive and multiplicative models for gamma distributed variables, and their application as psychometric models for response times. Psychometrika, 58, 445–469.
12.
MulderJ.van der LindenW. J. (2009). Multidimensional adaptive testing with optimal design criteria for item selection. Psychometrika, 74, 273–296.
13.
PrimiR. (2000). Complexity of geometric inductive reasoning tasks. Contribution to the understanding of fluid intelligence. Intelligence, 29, 41–70.
14.
RoskamE. E. (1997). Models for speed and time-limit tests. In van der LindenW. J.HambletonR. K. & (Eds.), Handbook of modern item response theory (pp. 187–208). New York, NY: Springer.
15.
ScheiblechnerH. (1979). Specific objective stochastic latency mechanisms. Journal of Mathematical Psychology, 19, 18–38.
16.
SegallD. O. (1996). Multidimensional adaptive testing. Psychometrika, 61, 331–354.
17.
SegallD. O. (2001). General ability measurement: An application of multidimensional item response theory. Psychometrika, 66, 79–97.
18.
ThissenD. (1983). Timed testing: An approach using item response theory. In WeissD. J. (Ed.), New horizons in testing: Latent trait test theory and computerized adaptive testing (pp. 179–203). New York, NY: Academic Press.
19.
van der LindenW. J. (1999a). Empirical initialization of the trait estimator in adaptive testing. Applied Psychological Measurement, 23, 21–29.
20.
van der LindenW. J. (1999b). Multidimensional adaptive testing with a minimum error-variance criterion. Journal of Educational and Behavioral Statistics, 24, 398–412.
21.
van der LindenW. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika,72, 287–308.
22.
van der LindenW. J. (2010). Statistical tests of conditional independence between responses and/or response times on test items. Psychometrika, 75, 120–139.
23.
van der LindenW. J.ScramsD. J.SchnipkeD. L. (1999). Using response-time constraints to control for speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195–210.
24.
WainerH. (2000). Computerized adaptive testing: A primer. Mahwah, NJ: Lawrence Erlbaum.
25.
WangC.ChangH.BoughtonK. A. (2011). Kullback-Leibler information and its application in multidimensional adaptive testing. Psychometrika, 76, 13–29.
26.
WangH.HansonB. (2005). Development and calibration of an Item Response Model that incorporates response time. Applied Psychological Measurement, 29, 323–339.
