Response time models (RTMs) are of increasing interest in educational and psychological testing. This article focuses on the lognormal model for response times, which is one of the most popular RTMs. Several existing statistics for testing normality and the fit of factor analysis models are repurposed for testing the fit of the lognormal model. A simulation study and two real data examples demonstrate the usefulness of the statistics. The Shapiro–Wilk test of normality and a z-test for factor analysis models were the most powerful in assessing the misfit of the lognormal model.
AdefisoyeJ. O.Golam KibriaB. M.GeorgeF. (2016). Performances of several univariate tests of normality: An empirical study. Journal of Biometrics & Biostatistics, 7(4), 1–8.
2.
American Educational Research Association, American Psychological Association, & National Council for Measurement in Education. (2014). Standards for educational and psychological testing. American Educational Research Association.
3.
AndersonT. W.DarlingD. A. (1954). A test of goodness of fit. Journal of the American Statistical Association, 49, 765–769.
4.
BockR. D.AitkinM. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.
5.
BolsinovaM.TijmstraJ. (2018). Improving precision of ability estimation: Getting more from response times. British Journal of Mathematical and Statistical Psychology, 71, 13–38.
6.
BoughtonK.SmithJ.RenH. (2017). Using response time data to detect compromised items and/or people. In CizekG. J.WollackJ. A. (Eds.), Handbook of detecting cheating on tests (pp. 177–190). Routledge.
7.
BoxG. E. P.DraperN. R. (1987). Empirical model-building and response surfaces. Wiley.
8.
CizekG. J.WollackJ. A. (2017). Handbook of detecting cheating on tests. Routledge.
9.
D’AgostinoR. B. (1971). An omnibus test of normality for moderate and large size samples. Biometrika, 58, 341.
10.
D’AgostinoR. B. (1986). Tests for the normal distribution. In D’AgostinoR. B.StephensM. A. (Eds.), Goodness-of-fit techniques (pp. 367–420). Marcel Dekker.
11.
De BoeckP.JeonM. (2019). An overview of models for response times and processes in cognitive tests. Frontiers in Psychology, 10, 1–11. http://doi.org/10.3389/fpsyg.2019.00102
12.
FingerM. S.CheeC. S. (2009, April). Response-time model estimation via confirmatory factor analysis[Paper presentation]. Annual meeting of the National Council of Measurement in Education, San Diego, CA, United States.
13.
FoxJ.-P.MariantiS. (2017). Person-fit statistics for joint models for accuracy and speed. Journal of Educational Measurement, 54, 243–262.
14.
GanF. F.KoehlerK. J. (1990). Goodness-of-fit tests based on p-p probability plots. Technometrics, 32, 289.
15.
GelmanA.CarlinJ. B.SternH. S.DunsonD. B.VehtariA.RubinD. B. (2014). Bayesian data analysis (3rd ed.). Chapman & Hall.
16.
GlasC. A. W.van der LindenW. J. (2010). Marginal likelihood inference for a model for item responses and response times. British Journal of Mathematical and Statistical Psychology, 63, 603–626.
JarqueC. M.BeraA. K. (1980). Efficient tests for normality, homoscedasticity and serial independence of regression residuals. Economics Letters, 6, 255–259.
19.
JoreskogK. G. (1967). Some contributions to maximum likelihood factor analysis. Psychometrika, 32, 443–482.
20.
Klein EntinkR. H.van der LindenW. J.FoxJ. P. (2009). A Box-Cox normal model for response times. British Journal of Mathematical and Statistical Psychology, 62, 621–640.
21.
KyllonenP.ZuJ. (2016). Use of response time for measuring cognitive ability. Journal of Intelligence, 4(14), 1–29.
22.
LeeY.-H.ChenH. (2011). A review of recent response-time analyses in educational testing. Psychological Test and Assessment Modeling, 53, 359–379.
23.
LillieforsH. W. (1967). On the Kolmogorov-Smirnov test for normality with mean and variance unknown. Journal of the American Statistical Association, 62, 399–402.
24.
MardiaK. V. (1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57, 519–530.
25.
MariantiS.FoxJ.-P.AvetisyanM.VeldkampB. P.TijmstraJ. (2014). Testing for aberrant behavior in response time modeling. Journal of Educational and Behavioral Statistics, 39, 426–451.
26.
MarisE. (1993). Additive and multiplicative models for gamma distributed random variables, and their application as psychometric models for response times. Psychometrika, 58, 445–469.
27.
MarisG.van der MaasH. (2012). Speed-accuracy response models: Scoring rules based on response time and accuracy. Psychometrika, 77, 615–633.
28.
Maydeu-OlivaresA. (2017). Assessing the size of model misfit in structural equation models. Psychometrika, 82, 533–558.
29.
MolenaarD.TuerlinckxF.van der MaasH. L. J. (2015). A bivariate generalized linear item response theory modeling framework to the analysis of responses and response times. Multivariate Behavioral Research, 50, 56–74.
30.
MooreD. S. (1986). Tests of chi-square type. In D’AgostinoR. B.StephensM. A. (Eds.), Goodness-of-fit techniques (pp. 63–96). Marcel Dekker.
31.
NikulinM. S. (1973). Chi-square test for continuous distributions with shift and scale parameters. Theory of Probability & Its Applications, 18, 559–568.
32.
OgasawaraH. (2001). Standard errors of fit indices using residuals in structural equation modeling. Psychometrika, 66, 421–436.
33.
OrlandoM.ThissenD. (2000). Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.
34.
PearsonK. (1900). On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling. Philosophical Magazine, 50, 157–175.
35.
QianH.StaniewskaD.ReckaseM.WooA. (2016). Using response time to detect item preknowledge in computer-based licensure examinations. Educational Measurement: Issues and Practice, 35(1), 38–47.
36.
RangerJ.KuhnJ. (2014). Testing fit of latent trait models for responses and response times in tests. Psychological Test and Assessment Modeling, 56, 382–404.
37.
RaoC. R. (1973). Linear statistical inference and its applications (2nd ed.). John Wiley.
38.
RaoK. C.RobsonD. S. (1974). A chi-square statistic for goodness-of-fit within the exponential family. Communications in Statistics, 3, 1139–1153.
39.
RaschG. (1960). Probabilistic models for some intelligence and attainment tests. Danish Institute for Educational Research.
40.
RazaliN. M.WahY. B. (2011). Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling tests. Journal of Statistical Modeling and Analytics, 2, 21–33.
41.
RosseelY. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.
42.
SatorraA. (1989). Alternative test criteria in covariance structure analysis: A unified approach. Psychometrika, 54, 131–151.
43.
ScheffeH. (1959). The analysis of variance. Wiley.
44.
SchnipkeD. L.ScramsD. J. (1997). Modeling item response times with a two-state mixture model: A new method of measuring speededness. Journal of Educational Measurement, 34, 213–232.
45.
SchnipkeD. L.ScramsD. J. (1999). Representing response-time information in item banks(LSAC-R-97-09). Law School Admission Council.
46.
SchnipkeD. L.ScramsD. J. (2002). Exploring issues of examinee behavior: Insights gained from response-time analyses. In MillsC.PotenzaM.FremerJ.WardW. (Eds.), Computer-based testing: Building the foundation for future assessments (pp. 237–266). Lawrence Erlbaum Associates.
47.
SeberG. A. F. (1984). Multivariate observations. John Wiley.
48.
ShapiroS. S.WilkM. B. (1965). An analysis of variance test for normality (complete samples). Biometrika, 52, 591.
49.
ShapiroS. S.WilkM. B.ChenH. J. (1968). A comparative study of various tests for normality. Journal of the American Statistical Association, 63, 1343–1372.
50.
SinghA. C. (1987). On the optimality and a generalization of Rao-Robson’s statistic. Communications in Statistics—Theory and Methods, 16, 3255–3273.
51.
SinharayS. (2018). A new person-fit statistic for the lognormal model for response times. Journal of Educational Measurement, 55, 457–476.
52.
SinharayS.HabermanS. J. (2014). How often is the misfit of item response theory models practically significant?Educational Measurement: Issues and Practice, 33(1), 23–35.
53.
SinharayS.JohnsonM. S. (2019). The use of item scores and response times to detect examinees who may have benefited from item preknowledge. British Journal of Mathematical and Statistical Psychology. http://doi.org/10.1111/bmsp.12187
54.
SkorupskiW. P.WainerH. (2017). The case for Bayesian methods when investigating test fraud. In CizekG. J.WollackJ. A. (Eds.), Handbook of detecting cheating on tests (pp. 214–231). Routledge.
55.
ThissenD. (1983). Timed testing: An approach using item response theory. In WeissD. J. (Ed.), New horizons in testing (pp. 179–203). Academic Press.
56.
ThodeH. (2002). Testing for normality. Marcel Dekker.
57.
van der LindenW. J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31, 181–204.
58.
van der LindenW. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287–308.
59.
van der LindenW. J. (2009). Conceptual issues in response-time modeling. Journal of Educational Measurement, 46, 247–272.
60.
van der LindenW. J.GlasC. A. W. (2010). Statistical tests of conditional independence between responses and/or response times on test items. Psychometrika, 75, 120–139.
61.
van der LindenW. J.GuoF. (2008). Bayesian procedures for identifying aberrant response-time patterns in adaptive testing. Psychometrika, 73, 365–384.
62.
van der LindenW. J.Klein EntinkR. H.FoxJ.-P. (2010). IRT parameter estimation with response times as collateral information. Applied Psychological Measurement, 34, 327–347.
63.
van der MaasH. L. J.MolenaarD.MarisG.KievitR. A.BorsboomD. (2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118, 339–356.
64.
van RijnP. W.AliU. S. (2017). A comparison of item response models for accuracy and speed of item responses with applications to adaptive testing. British Journal of Mathematical and Statistical Psychology, 70, 317–345.
65.
VoinovV.NikulinM.BalakrishnanN. (2013). Chi-squared goodness of fit tests with applications. Elsevier.
66.
VoinovV.PyaN.AlloyarovaR. (2009). A comparative study of some modified chi-squared tests. Communications in Statistics—Simulation and Computation, 38, 355–367.
67.
WangC.XuG.ShangZ.KuncelN. (2018). Detecting aberrant behavior and item preknowledge: A comparison of mixture modeling method and residual method. Journal of Educational and Behavioral Statistics, 43, 469–501.
68.
WangT.HansonB. A. (2005). Development and calibration of an item response model that incorporates response time. Applied Psychological Measurement, 29, 323–339.
69.
WeiT.SimkoV. (2017). R package “corrplot”: Visualization of a correlation matrix [Version 0.84]. Available fromhttps://github.com/taiyun/corrplot.
70.
WeisbergS.BinghamC. (1975). An approximate analysis of variance test for non-normality suitable for machine calculation. Technometrics, 17, 133.
71.
WiseS. L.KongX. (2005). Response time effort: A new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18, 163–183.
72.
WollackJ. A.SchoenigR. W. (2018). Cheating. In FreyB. B. (Ed.), The SAGE encyclopedia of educational research, measurement, and evaluation (pp. 260–265). Sage.
73.
YaziciB.YolacanS. (2007). A comparison of various tests of normality. Journal of Statistical Computation and Simulation, 77, 175–183.
74.
YenW. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145.
