The hierarchical model of van der Linden is the most popular model for responses and response times in tests. It is composed of two separate submodels—one for the responses and one for the response times—that are joined at a higher level. The submodel for the response times is based on the lognormal distribution. The lognormal distribution is a skew distribution with a support from zero to infinity. Such a support is unrealistic as the solution process demands a minimal processing time that sets a response time threshold. Ignoring this response time threshold misspecifies the model and threatens the validity of model-based inferences. In this article, the response time model of van der Linden is replaced by a model that is based on the three-parameter lognormal distribution. The three-parameter lognormal distribution extends the lognormal distribution by an additional location parameter that bounds the support away from zero. Two different approaches to model fitting are proposed and evaluated with regard to parameter recovery in a simulation study. The extended model is applied to two data sets. In both data sets, the extension improves the fit of the hierarchical model.
AkaikeH. (1992). Information theory and an extension of the maximum likelihood principle. In KotzS.JohnsonN. (Eds.), Breakthroughs in statistics: Vol. I. Foundations and basic theory (pp. 610-624). Springer.
2.
AlqallafF.van AelstS.YohaiV.ZamarR. (2009). Propagation of outliers in multivariate data. Annals of Statistics, 37(1), 311-331. https://doi.org/10.1214/07-AOS588
3.
AristizabalR.ChenZ. (2012). Estimating parameters of the three-parameter lognormal distribution. Lambert Academic.
BolsinovaM.TijmstraJ. (2018). Improving precision of ability estimation: Getting more from response times. British Journal of Mathematical and Statistical Psychology, 71(1), 13-38. https://doi.org/10.1111/bmsp.12104
CattellR.WarburtonF. (1967). Objective personality and motivation tests: A theoretical introduction and practical compendium. University of Illinois Press.
8.
ChanS.-C.LuT.-S.TsaiR.-C. (2014). Incorporating response time to analyze test data with mixture structural equation modeling. Psychological Testing, 61, 463-488.
9.
ChengR.AminN. (1983). Estimating parameters in continuous univariate distributions with a shifted origin. Journal of the Royal Statistical Society, B, 45(3), 394-403. https://doi.org/10.1111/j.2517-6161.1983.tb01268.x
10.
CizekG.WollackJ. (2017). Exploring cheating on tests: The context, the concern, and the challenges. In CizekG.WollackJ. (Eds.), Handbook of quantitative methods for detecting cheating on tests (pp. 3-19). Routledge. https://doi.org/10.4324/9781315743097
11.
CohenA. (1987). Three-parameter estimation. In CrowE.ShimizuK. (Eds.), Lognormal distributions: Theory and applications (pp. 113-137). CRC Press.
12.
CohenA.WhittenB. (1980). Estimation in the three-parameter lognormal distribution. Journal of the American Statistical Association, 75(370), 107-122. https://doi.org/10.2307/2287466
13.
CosineauD. (2009). Fitting the three-parameter Weibull distribution: Review and evaluation of existing and new methods. IEEE Transactions on Dielectrics and Electrical Insulation, 16(1), 281-288. https://doi.org/10.1109/TDEI.2009.4784578
14.
GaviriaJ. (2005). Increase in precision when estimating parameters in computer assisted testing using response time. Quality & Quantity, 39(1), 45-69. https://doi.org/10.1007/s11135-004-0437-y
15.
GiesbrechtF.KempthorneO. (1976). Maximum likelihood estimation in the three-parameter lognormal distribution. Journal of the Royal Statistical Society, 38(3), 257-264. https://doi.org/10.1111/j.2517-6161.1976.tb01591.x
16.
GlasC.van der LindenW. (2010). Marginal likelihood inference for a model for item responses and response times. British Journal of Mathematical and Statistical Psychology, 63(3), 603-626. https://doi.org/10.1348/000711009X481360
17.
GoegebeurY.De BoeckP.WollackJ.CohenA. (2008). A speeded item response model with gradual process change. Psychometrika, 73(1), 65-87. https://doi.org/10.1007/S11336-007-9031-2
18.
GongG.SamaniegoF. (1981). Pseudo maximum likelihood estimation: Theory and applications. Annals of Statistics, 9(4), 861-869. https://doi.org/10.1214/aos/1176345526
19.
HarterH.MooreA. (1966). Local-maximum-likelihood estimation of the parameters of three-parameter lognormal populations from complete and censored samples. Journal of the American Statistical Association, 61(315), 842-851. https://doi.org/10.2307/2282793
20.
HeathcoteA.BrownS.CousineauD. (2004). QMPE: Estimating lognormal, Wald, and Weibull RT distributions with a parameter-dependent lower bound. Behavior Research Methods, Instruments, & Computers, 36(2), 277-290. https://doi.org/10.3758/BF03195574
21.
HillB. (1963). The three-parameter lognormal distribution and Bayesian analysis of a point-source epidemic. Journal of the American Statistical Association, 58(301), 72-84. https://doi.org/10.2307/2282955
22.
HuberP. (1981). Robust statistics. Wiley.
23.
KleinJ.MoeschbergerM. (2003). Survival analysis: Techniques for censored and truncated data. Springer. https://doi.org/10.1007/b97377
24.
KochT.OrtnerT. M.EidM.CaspersJ.SchmittM. (2014). Evaluating the construct validity of objective personality tests using a multitrait-multimethod-multioccasion-(MTMM-MO)-approach. European Journal of Psychological Assessment, 30(3), 208-230. https://doi.org/10.1027/1015-5759/a000212
25.
Le CamL. (1990). Maximum likelihood: An introduction. International Statistical Review, 58(2), 153-171. https://doi.org/10.2307/1403464
26.
MariantiS.FoxJ.-P.AvetisyanM.VeldkampM.TijmstraJ. (2014). Testing for aberrant behavior in response time modeling. Journal of Educational and Behavioral Statistics, 39(6), 426-451. https://doi.org/10.3102/1076998614559412
McDonaldR. P. (1985). Factor analysis and related methods. Lawrence Erlbaum.
29.
MolenaarD.TuerlinckxF.van der MaasH. (2015a). A bivariate generalized linear item response theory modeling framework to the analysis of responses and response times. Multivariate Behavioral Research, 50(1), 56-74. https://doi.org/10.1080/00273171.2014.962684
30.
MolenaarD.TuerlinckxF.van der MaasH. (2015b). A generalized linear factor model approach to the hierarchical framework for responses and response times. British Journal of Mathematical and Statistical Psychology, 68(2), 197-219. https://doi.org/10.1111/bmsp.12042
OrtnerT. M.ProyerR. (2018). Behavioral and performance measures of personality. In Zeigler-HillV.ShackelfordT. (Eds.), Encyclopedia of personality and individual differences (pp. 1-6). Springer. https://doi.org/10.1007/978-3-319-28099-8_1281-1
33.
ParkeW. (1986). Pseudo maximum likelihood estimation: The asymptotic distribution. Annals of Statistics, 14(1), 355-357. https://doi.org/10.1214/aos/1176349862
34.
R Development Core Team. (2009). R: A language and environment for statistical computing [Computer software manual]. http://www.R-project.org
35.
RangerJ.KuhnJ. (2013). Analyzing response times with rank correlation approaches. Journal of Educational and Behavioral Statistics, 38(1), 61-80. https://doi.org/10.3102/1076998611431086
36.
RangerJ.WolgastA. (2019). Using response times as collateral information about latent traits in psychological tests. Methodology, 15(4), 185-196. https://doi.org/10.1027/1614-2241/a000181
SelfS.LiangK. (1987). Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. Journal of the American Statistical Association, 82(398), 605-610. https://doi.org/10.2307/2289471
39.
SinharayS. (2018). A new person-fit statistic for the lognormal model for response times. Journal of Educational Measurement, 55(4), 457-476. https://doi.org/10.1111/jedm.12188
40.
SinharayS.HabermanS. (2014). How often is the misfit of item response theory models practically significant. Educational Measurement: Issues and Practice, 33(1), 23-35. https://doi.org/10.1111/emip.12024
41.
SmithR. (1985). Maximum likelihood estimation in a class of nonregular cases. Biometrika, 72(1), 67-90. https://doi.org/10.2307/2336336
UlrichR.MillerJ. (1994). Effects of truncation on reaction time analysis. Journal of Experimental Psychology: General, 123(1), 34-80. https://doi.org/10.1037/0096-3445.123.1.34
44.
van der LindenW. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31(2), 181-204. https://doi.org/10.3102/10769986031002181
45.
van der LindenW. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287-308. https://doi.org/10.1007/s11336-006-1478-z
46.
van der LindenW. (2009). A bivariate lognormal response-time model for the detection of collusion between test takers. Journal of Educational and Behavioral Statistics, 34(3), 378-394. https://doi.org/10.3102/1076998609332107
47.
van der LindenW.GuoF. (2008). Bayesian procedures for identifying aberrant response time patterns in adaptive testing. Psychometrika, 73(3), 365-384. https://doi.org/10.1007/s11336-007-9046-8
48.
van der LindenW.Klein EntinkR.FoxJ. (2010). IRT parameter estimation with response times as collateral information. Applied Psychological Measurement, 34, 327–347. https://doi.org/10.1177/0146621609349800
49.
WangC.ChangH.DouglasJ. (2013). The linear transformation model with frailties for the analysis of item response times. British Journal of Mathematical and Statistical Psychology, 66(1), 144-168. https://doi.org/10.1111/j.2044-8317.2012.02045.x
50.
WangC.FanZ.ChangH.DouglasJ. (2013). A semiparametric model for jointly analyzing response times and accuracy in computerized testing. Journal of Educational and Behavioral Statistics, 38(4), 381-417. https://doi.org/10.3102/1076998612461831
51.
WangC.XuG.ShangZ. (2018). A two-stage approach to differentiating normal and aberrant behavior in computer based testing. Psychometrika, 83(1), 223-254. https://doi.org/10.1007/s11336-016-9525-x
52.
WangH.FlournoyN. (2015). On the consistency of the maximum likelihood estimator for the three parameter lognormal distribution. Statistics & Probability Letters, 105, 57-64. https://doi.org/10.1016/j.spl.2015.05.021
53.
WhiteH. (1982). Maximum likelihood estimation of misspecified models. Econometrica, 50(1), 1-25. https://doi.org/10.2307/1912526
54.
WiseS. (2017). Rapid-guessing behavior: Its identification, interpretation, and implications. Educational Measurement, 36(4), 52-61. https://doi.org/10.1111/emip.12165
ZopluogluC. (2019). Detecting examinees with item preknowledge in large-scale testing using extreme gradient boosting (XGBoost). Educational and Psychological Measurement, 79(5), 931-961. https://doi.org/10.1177/0013164419839439
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
