The speed–accuracy trade-off (SAT) suggests that time constraints reduce response accuracy. Its relevance in observational settings—where response time (RT) may not be constrained but respondent speed may still vary—is unclear. Using 29 data sets containing data from cognitive tasks, we use a flexible method for identification of the SAT (which we test in extensive simulation studies) to probe whether the SAT holds. We find inconsistent relationships between time and accuracy; marginal increases in time use for an individual do not necessarily predict increases in accuracy. Additionally, the speed–accuracy relationship may depend on the underlying difficulty of the interaction. We also consider the analysis of items and individuals; of particular interest is the observation that respondents who exhibit more within-person variation in response speed are typically of lower ability. We further find that RT is typically a weak predictor of response accuracy. Our findings document a range of empirical phenomena that should inform future modeling of RTs collected in observational settings.
BergéL. (2018). Efficient estimation of maximum likelihood models with multiple fixed-effects: The r package fenmlm (tech. rep.). Center for Research in Economic Analysis, University of Luxembourg.
2.
BolsinovaM.MolenaarD. (2018). Modeling nonlinear conditional dependence between response time and accuracy. Frontiers in psychology, 9, 1525.
3.
ChalmersR. P. (2012). Mirt: A multidimensional item response theory package for the r environment. Journal of Statistical Software, 48(6), 1–29.
4.
ChenH.De BoeckP.GradyM.YangC.-L.WaldschmidtD. (2018). Curvilinear dependency of response accuracy on response time in cognitive tests. Intelligence, 69, 16–23.
5.
CheungY. B. (2007). A modified least–squares regression approach to the estimation of risk difference. American Journal of Epidemiology, 166(11), 1337–1344.
6.
DavidsonM. C.AmsoD.AndersonL. C.DiamondA. (2006). Development of cognitive control and executive functions from 4 to 13 years: Evidence from manipulations of memory, inhibition, and task switching. Neuropsychologia, 44(11), 2037–2078.
7.
DebeerD.JanssenR. (2013). Modeling item–position effects within an IRT framework. Journal of Educational Measurement, 50(2), 164–185.
8.
De BoeckP.BakkerM.ZwitserR.NivardM.HofmanA.TuerlinckxF.PartchevI. (2011). The estimation of item response models with the lmer function from the lme4 package in r. Journal of Statistical Software, 39(12), 1–28.
9.
De BoeckP.JeonM. (2019). An overview of models for response times and processes in cognitive tests. Frontiers in Psychology, 10, 102.
10.
DennisI.EvansJ. S. B. (1996). The speed–error trade-off problem in psychometric testing. British Journal of Psychology, 87(1), 105–129.
11.
DomingueB. W.KanopkaK.StenhaugB.SolandJ.KuhfeldM.WiseS.PiechC. (2021). Variation in respondent speed and its implications: Evidence from an adaptive testing scenario. Journal of Educational Measurement, 58(3), 335–363.
12.
DraheimC.MashburnC. A.MartinJ. D.EngleR. W. (2019). Reaction time in differential and developmental research: A review and commentary on the problems and alternatives. Psychological Bulletin, 145(5), 508.
13.
DraheimC.TsukaharaJ. S.MartinJ. D.MashburnC. A.EngleR. W. (2020). A toolbox approach to improving the measurement of attention control. Journal of Experimental Psychology: General, 150(2), 242–275.
14.
FifeD. A.RodgersJ. L. (2021). Understanding the exploratory/confirmatory data analysis continuum: Moving beyond the “replication crisis.”American Psychologist. doi: https://doi.org/10.1037/amp0000886
15.
FriedmanJ.HastieT.TibshiraniR. (2001). The elements of statistical learning (Vol. 1). Springer series in statistics.
16.
GoldhammerF.NaumannJ.StelterA.TóthK.RölkeH.KliemeE. (2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106(3), 608.
17.
HallionL. S.KusmierskiS. N.CaulfieldM. K. (2020). Worry alters speed-accuracy tradeoffs but does not impair sustained attention. Behaviour Research and Therapy, 128, 103597.
18.
HeitzR. P. (2014). The speed–accuracy tradeoff: History, physiology, methodology, and behavior. Frontiers in Neuroscience, 8, 150.
19.
HughesM. M.LinckJ. A.BowlesA. R.KoethJ. T.BuntingM. F. (2014). Alternatives to switch-cost scoring in the task-switching paradigm: Their reliability and increased validity. Behavior Research Methods, 46(3), 702–721.
20.
JaccardJ.BrinbergM. (2021). Monte Carlo simulations using extant data to mimic populations: Applications to the modified linear probability model and logistic regression. Psychological Methods, 26(4), 450–465.
21.
KangI.De BoeckP.RatcliffR. (2021). Modeling conditional dependence of response accuracy and response time with the diffusion item response theory model. doi: https://doi.org/10.1007/s11336-021-09819-5
22.
LappinJ. S.DischK. (1972). The latency operating characteristic: Ii. effects of visual stimulus intensity on choice reaction time. Journal of Experimental Psychology, 93(2), 367.
23.
LiesefeldH. R.JanczykM. (2019). Combining speed and accuracy to control for speed–accuracy trade-offs(?). Behavior Research Methods, 51(1), 40–60.
24.
LövdénM.LiS.-C.ShingY. L.LindenbergerU. (2007). Within-person trial-to-trial variability pre-cedes and predicts cognitive decline in old and very old age: Longitudinal data from the berlin aging study. Neuropsychologia, 45(12), 2827–2838.
25.
MarisG.Van der MaasH. (2012). Speed-accuracy response models: Scoring rules based on response time and accuracy. Psychometrika, 77(4), 615–633.
26.
MolenaarD.BolsinovaM.VermuntJ. K. (2018). A semi–parametric within–subject mixture approach to the analyses of responses and response times. British Journal of Mathematical and Statistical Psychology, 71(2), 205–228.
27.
MolenaarD.TuerlinckxF.van der MaasH. L.Etal. (2015). Fitting diffusion item response theory models for responses and response times using the r package diffirt. Journal of Statistical Software, 66(4), 1–34.
28.
PachellaR. G.FisherD. F.KarshR. (1968). Absolute judgments in speeded tasks: Quantification of the trade-off between speed and accuracy. Psychonomic Science, 12(6), 225–226.
29.
PalmerJ.HukA. C.ShadlenM. N. (2005). The effect of stimulus strength on the speed and accuracy of a perceptual decision. Journal of Vision, 5(5), 1–1.
30.
PewR. W. (1969). The speed–accuracy operating characteristic. Acta Psychologica, 30, 16–26.
31.
RangerJ.KuhnJ.GaviriaJ.-L. (2015). A race model for responses and response times in tests. Psychometrika, 80(3), 791–810.
32.
RangerJ.KuhnJ.PohlS. (2021). Effects of motivation on the accuracy and speed of responding in tests: The speed-accuracy tradeoff revisited. Measurement: Interdisciplinary Research and Perspectives, 19(1), 15–38.
33.
RaschG. (1993). Probabilistic models for some intelligence and attainment tests. ERIC.
34.
RatcliffR.SmithP. L.BrownS. D.McKoonG. (2016). Diffusion decision model: Current issues and history. Trends in Cognitive Sciences, 20(4), 260–281.
35.
SalthouseT. A. (1979). Adult age and the speed-accuracy trade-off. Ergonomics, 22(7), 811–821.
36.
SchoutenJ.BekkerJ. (1967). Reaction time and accuracy. Acta Psychologica, 27, 143–153.
37.
SimonsD. J. (2000). Attentional capture and inattentional blindness. Trends in Cognitive Sciences, 4(4), 147–155.
38.
SmallwoodJSchoolerJ. W. (2015). The science of mind wandering: Empirically navigating the stream of consciousness. Annual Review of Psychology, 66, 487–518.
39.
SuS.DavisonM. L. (2019). Improving the predictive validity of reading comprehension using response times of correct item responses. Applied Measurement in Education, 32(2), 166–182.
40.
Tucker–DrobE. M. (2019). Cognitive aging and dementia: A life–span perspective. Annual Review of Developmental Psychology, 1, 177–196.
41.
TukeyJ. W. (1977). Exploratory data analysis (Vol. 2). Reading, Mass.
42.
van der LindenW. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287.
43.
VandierendonckA. (2017). A comparison of methods to combine speed and accuracy measures of performance: A rejoinder on the binning procedure. Behavior Research Methods, 49(2), 653–673.
44.
VandierendonckA. (2018). Further tests of the utility of integrated speed–accuracy measures in task switching. Journal of Cognition, 1(1): 8.
45.
van RijnP. W.AliU. S. (2018). A generalized speed–accuracy response model for dichotomous items. Psychometrika, 83(1), 109–131.
46.
VidaL. J.BrinkhuisM. J. S.BolsinovaM. (2021). Speeding up without loss of accuracy: Item position effects on performance in university exams. Proceedings of the 14th International Conference on Educational Data Mining. EDM 2021, Educational Data Mining.
47.
WangC.XuG. (2015). A mixture hierarchical model for response times and response accuracy. British Journal of Mathematical and Statistical Psychology, 68(3), 456–477.
48.
WickelgrenW. A. (1977). Speed–accuracy tradeoff and information processing dynamics. Acta Psychologica, 41(1), 67–85.
49.
WiseS. L. (2015). Response time as an indicator of test taker speed: Assumptions meet reality. Measurement: Interdisciplinary Research and Perspectives, 13(3–4), 186–188.
50.
WoodsC. M.ThissenD. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71(2), 281–301.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
