An adaptive testing method is presented that controls the speededness of a test using predictions of the test takers' response times on the candidate items in the pool. Two different types of predictions are investigated: posterior predictions given the actual response times on the items already administered and posterior predictions that use the responses on these items as an additional source of information. In a simulation study with an adaptive test modeled after a test from the Armed Services Vocational Aptitude Battery, the effectiveness of the methods in removing differential speededness from the test was evaluated.
Bejar, I.I. (1985). Test speededness under number-right scoring: An analysis of the Test of English as a Foreign Language (Research Report No. 85-11). Princeton, NJ: Educational Testing Service.
2.
Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In F. M. Lord & M. R. Novick (Eds.), Statistical theories of mental test scores (pp. 397-479). Reading, MA : Addison-Wesley.
3.
Bridgeman, B., & Cline, F. (2004). Effects of differentially time-consuming tests on computerized-adaptive test scores. Journal of Educational Measurement, 41, 137-148.
4.
Chang, H.-H., & Stout, W. (1993). The asymptotic posterior normality of the latent trait in an IRT model. Psychometrika, 58, 37-52.
5.
Evans, F.R., & Reilly, R.R. (1972). A study of test speededness as a source of bias . Journal of Educational Measurement, 9, 123-131.
6.
Gelman, A., Carlin, J.B., Stern, H., & Rubin, D.B. (1995). Bayesian data analysis. London : Chap-man & Hall.
7.
ILOG, Inc. (2003). CPLEX 9.0 [Computer software]. Incline Village, NV: Author.
8.
Kingston, N.M., & Dorans, N.J. (1984). Item location effects and their implications for IRT equating and adaptive testing. Applied Psychological Measurement , 8, 147-154.
9.
Luce, R.D. (1986). Response times: Their roles in inferring elementary mental organization. Oxford, UK: Oxford University Press.
10.
Oshima, T.C. (1994). The effect of speededness on parameter estimation in item response theory. Journal of Educational Measurement , 31, 200-219.
11.
Owen, R.J. (1975). A Bayesian sequential procedure for quantal response in the context of adaptive mental testing. Journal of the American Statistical Association, 70, 351-356.
12.
van der Linden, W.J. (2005). Linear models for optimal test assembly. New York: Springer.
13.
van der Linden, W.J. (2006). A lognormal model for response times on test items . Journal of Educational and Behavioral Statistics, 31, 181-204.
14.
van der Linden, W.J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287-308.
15.
van der Linden, W.J. (2008). Using response times for item selection in adaptive tests. Journal of Educational and Behavioral Statistics, 31, 5-20.
16.
van der Linden, W.J., Breithaupt, K., Chuah, S.C., & Zhang, Y. (2007). Detecting differential speededness in multistage testing. Journal of Educational Measurement, 44, 117-130.
17.
van der Linden, W.J., & Guo, F. (2008). Bayesian procedures for identifying aberrant response-time patterns in adaptive testing. Psychometrika. Published online January 9, 2008; DOI: 10.1007/ s11336-007-9046-8
18.
van der Linden, W.J., Scrams, D.J., & Schnipke, D.L. (1999). Using response-time constraints to control for speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195-210.
19.
Wollack, J.A., Cohen, A.S., & Wells, C.S. (2003). A method of maintaining scale stability in the presence of test speededness. Journal of Educational Measurement , 40, 307-330.
20.
Yamamoto, K., & Everson, H. (1997). Modeling the effects of test length and test time on parameter estimation using the HYBRID model. In J. Rost & R. Langeheine (Eds.), Applications of latent trait and latent class models in the social sciences (pp. 69-99). New York: John Wiley.
