Interference tasks (e.g., the Stroop task) exhibit low test–retest reliability, which hinders the measurement of individual differences. This study examined whether controlling the interference rate improves reliability. The interference rate is a metric derived from the Horse-Race Model, defined as the probability of experiencing semantic-color conflict in incongruent Stroop trials (distinct from list-wide congruency, i.e., the proportion of incongruent trials in a block). In Studies 1a and 1b, interference rate was positively correlated with the magnitude of the Stroop effect, indicating that it accounts for meaningful individual variability. In Study 2, the fixed-interference-rate group (IR-fixed) had their interference rate controlled at 75% across two testing sessions (2-week interval) by adjusting the stimulus-onset asynchrony between semantic and color features (i.e., ensuring a 75% probability of experiencing interference in incongruent trials). Compared to the random-interference-rate group (IR-random), the IR-fixed group showed significantly higher test–retest reliability and stronger cross-session predictability of the Stroop effect, demonstrating that controlling the interference rate enhances measurement stability. Studies 1b and 2 also revealed moderate cross-session correlations for interference rate, regardless of whether the interval was a single Stroop task administration or 2 weeks. Furthermore, across all studies, interference rate exhibited poor internal consistency (McDonald’s ω < .60). Overall, the results suggest that interference rate can predict the Stroop effect but is not a stable psychological trait; instead, it fluctuates within a certain range. Therefore, failing to control the interference rate adversely impacts the test–retest reliability of the Stroop task.
AlgomD.ChajutE. (2019). Reclaiming the Stroop effect back from control to input-driven attention and perception. Frontiers in Psychology, 10, 1683. https://doi.org/10.3389/fpsyg.2019.01683
2.
BraemS.BuggJ. M.SchmidtJ. R.CrumpM. J. C.WeissmanD. H.NotebaertW.EgnerT. (2019). Measuring adaptive control in conflict tasks. Trends in Cognitive Sciences, 23(9), 769–783. https://doi.org/10.1016/j.tics.2019.07.002
3.
BurgoyneA. P.TsukaharaJ. S.MashburnC. A.PakR.EngleR. W. (2023). Nature and measurement of attention control. Journal of Experimental Psychology: General, 152(8), 2369–2402. https://doi.org/10.1037/xge0001408
4.
ChenA.BaileyK.TiernanB. N.WestR. (2011). Neural correlates of stimulus and response interference in a 2–1 mapping stroop task. International Journal of Psychophysiology, 80(2), 129–138. https://doi.org/10.1016/j.ijpsycho.2011.02.012
5.
CohenJ. D.DunbarK.McClellandJ. L. (1990). On the control of automatic processes: A parallel distributed processing account of the Stroop effect. Psychological Review, 97(3), 332–361. https://doi.org/10.1037/0033-295X.97.3.332
DraheimC.HicksK. L.EngleR. W. (2016). Combining reaction time and accuracy: The relationship between working memory capacity and task switching as a case example. Perspectives on Psychological Science, 11(1), 133–155. https://doi.org/10.1177/1745691615596990
10.
DraheimC.MashburnC. A.MartinJ. D.EngleR. W. (2019). Reaction time in differential and developmental research: A review and commentary on the problems and alternatives. Psychological Bulletin, 145(5), 508–535. https://doi.org/10.1037/bul0000192
11.
DraheimC.TshukaraJ. S.EngleR. W. (2023). Replication and extension of the toolbox approach to measuring attention control. Behavior Research Methods, 56(3), 2135–2157. https://doi.org/10.3758/s13428-023-02140-2
12.
DraheimC.TsukaharaJ. S.MartinJ. D.MashburnC. A.EngleR. W. (2021). A toolbox approach to improving the measurement of attention control. Journal of Experimental Psychology: General, 150(2), 242–275. https://doi.org/10.1037/xge0000783
13.
DunbarK.MacLeodC. M. (1984). A horse race of a different color: Stroop interference patterns with transformed words. Journal of Experimental Psychology: Human Perception and Performance, 10(5), 622–639. https://doi.org/10.1037/0096-1523.10.5.622
14.
DyerF. N. (1971). The duration of word meaning responses: Stroop interference for different preexposures of the word. Psychonomic Science, 25(4), 229–231. https://doi.org/10.3758/BF03329102
15.
DyerF. N. (1973). The Stroop phenomenon and its use in the stlldy of perceptual, cognitive, and response processes. Memory & Cognition, 1(2), 106–120. https://doi.org/10.3758/BF03198078
EppA. M.DobsonK. S.DozoisD. J. A.FrewenP. A. (2012). A systematic meta-analysis of the Stroop task in depression. Clinical Psychology Review, 32(4), 316–328. https://doi.org/10.1016/j.cpr.2012.02.005
20.
EriksenB. A.EriksenC. W. (1974). Effects of noise letters upon the identification of a target letter in a nonsearch task. Perception & Psychophysics, 16(1), 143–149. https://doi.org/10.3758/BF03203267
21.
EysenckM. W.DerakshanN. (2011). New perspectives in attentional control theory. Personality and Individual Differences, 50(7), 955–960. https://doi.org/10.1016/j.paid.2010.08.019
FaulF.ErdfelderE.BuchnerA.LangA.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41(4), 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
24.
FriedmanN. P.MiyakeA. (2004). The relations among inhibition and interference control functions: A latent-variable analysis. Journal of Experimental Psychology: General, 133(1), 101–135. https://doi.org/10.1037/0096-3445.133.1.101
25.
GärtnerA.StrobelA. (2021). Individual differences in inhibitory control: A latent variable analysis. Journal of Cognition, 4(1), 17. https://doi.org/10.5334/joc.150
26.
GengJ.SchnurT. T.JanssenN. (2014). Relative speed of processing affects interference in Stroop and picture–word interference paradigms: Evidence from the distractor frequency effect. Language, Cognition and Neuroscience, 29(9), 1100–1114. https://doi.org/10.1080/01690965.2013.846473
27.
GlaserM. O.GlaserW. R. (1982). Time course analysis of the Stroop phenomenon. Journal of Experimental Psychology: Human Perception and Performance, 8(6), 875–894. https://doi.org/10.1037/0096-1523.8.6.875
28.
HedgeC.PowellG.BompasA.SumnerP. (2022). Strategy and processing speed eclipse individual differences in control ability in conflict tasks. Journal of Experimental Psychology: Learning, Memory, and Cognition, 48(10), 1448–1469. https://doi.org/10.1037/xlm0001028
29.
HedgeC.PowellG.SumnerP. (2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186. https://doi.org/10.3758/s13428-017-0935-1
30.
HommelB. (1993). The relationship between stimulus processing and response selection in the Simon task: Evidence for a temporal overlap. Psychological Research, 55(4), 280–290. https://doi.org/10.1007/BF00419688
31.
HuangL.LiB.YanP.XuC.TianX.HanC.LiX. (2022). Using relative-speed-of-processing to explain the shielding function of task rules. Quarterly Journal of Experimental Psychology, 75(11), 2177–2196. https://doi.org/10.1177/17470218211069484
32.
KucinaT.WellsL.LewisI.De SalasK.KohlA.PalmerM. A.SauerJ. D.MatzkeD.AidmanE.HeathcoteA. (2023). Calibration of cognitive tests to address the reliability paradox for decision-conflict tasks. Nature Communications, 14(1), 2234. https://doi.org/10.1038/s41467-023-37777-2
33.
LavieN. (1995). Perceptual load as a necessary condition for selective attention. Journal of Experimental Psychology: Human Perception and Performance, 21(3), 451–468. https://doi.org/10.1037/0096-1523.21.3.451
34.
LoganG. D.ZbrodoffN. J. (1998). Stroop-type interference: Congruity effects in color naming with typewritten responses. Journal of Experimental Psychology: Human Perception and Performance, 24(3), 978–992. https://doi.org/10.1037/0096-1523.24.3.978
35.
LuoC. R. (1999). Semantic competition as the basis of Stroop interference: Evidence from color-word matching tasks. Psychological Science, 10(1), 35–40. https://doi.org/10.1111/1467-9280.00103
McGrawK. O.WongS. P. (1996). “Forming inferences about some intraclass correlations coefficients”: Correction. Psychological Methods, 1(4), 390–390. https://doi.org/10.1037/1082-989X.1.4.390
39.
MorettiL.KochI.HornjakR.Von BastianC. C. (2025). Quality over quantity: Focusing on high-conflict trials to improve the reliability and validity of attentional control measures. Journal of Experimental Psychology: Learning, Memory, and Cognition, 52(1), 34–54. https://doi.org/10.1037/xlm0001466
40.
MorettiL.KochI.SteinhauserM.SchuchS. (2023). Disentangling task-selection failures from task-execution failures in task switching: An assessment of different paradigms. Psychological Research, 87(3), 929–950. https://doi.org/10.1007/s00426-022-01708-5
41.
NakagawaS.SchielzethH. (2010). Repeatability for Gaussian and non-Gaussian data: A practical guide for biologists. Biological Reviews, 85(4), 935–956. https://doi.org/10.1111/j.1469-185X.2010.00141.x
42.
ParrisB. A.HasshimN.WadsleyM.AugustinovaM.FerrandL. (2022). The loci of Stroop effects: A critical review of methods and evidence for levels of processing contributing to color-word Stroop effects and the implications for the loci of attentional selection. Psychological Research, 86(4), 1029–1053. https://doi.org/10.1007/s00426-021-01554-x
43.
PosnerM. I.SnyderC. R. R. (2004). Attention and cognitive control. In BalotaD. A.MarshE. J. (Eds.), Cognitive psychology: Key readings (pp. 205–223). Psychology Press.
44.
ProvostS. C.WoodwardR. (1991). Effects of nicotine gum on repeated administration of the stroop test. Psychopharmacology, 104(4), 536–540. https://doi.org/10.1007/BF02245662
45.
R Core Team (2023). R: A language and environment for statistical computing (Version 4.2.3). R Foundation for Statistical Computing. https://www.R-project.org/
46.
RangerJ.KuhnJ. T.OrtnerT. M. (2020). Modeling responses and response times in tests with the hierarchical model and the three-parameter lognormal distribution. Educational and Psychological Measurement, 80(6), 1059–1089. https://doi.org/10.1177/0013164420908916
47.
Rey-MermetA.GadeM.OberauerK. (2018). Should we stop thinking about inhibition? Searching for individual and age differences in inhibition ability. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(4), 501–526. https://doi.org/10.1037/xlm0000450
48.
Rey-MermetA.GadeM.SouzaA. S.Von BastianC. C.OberauerK. (2019). Is executive control related to working memory capacity and fluid intelligence?Journal of Experimental Psychology: General, 148(8), 1335–1372. https://doi.org/10.1037/xge0000593
49.
RothermundK.GollnickN.GiesenC. G. (2022). Accounting for proportion congruency effects in the Stroop task in a confounded setup: Retrieval of stimulus-response episodes explains it all. Journal of Cognition, 5(1), 39. https://doi.org/10.5334/joc.232
50.
RouderJ. N.HaafJ. M. (2019). A psychometrics of individual differences in experimental tasks. Psychonomic Bulletin & Review, 26(2), 452–467. https://doi.org/10.3758/s13423-018-1558-y
51.
RouderJ. N.KumarA.HaafJ. M. (2023). Why many studies of individual differences with inhibition tasks may not localize correlations. Psychonomic Bulletin & Review, 30(6), 2049–2066. https://doi.org/10.3758/s13423-023-02293-3
52.
ScheibeK. E.ShaverP. R.CarrierS. C. (1967). Color association values and response interference on variants of the Stroop test. Acta Psychologica, 26, 286–295. https://doi.org/10.1016/0001-6918(67)90028-5
53.
SchönbrodtF. D.PeruginiM. (2013). At what sample size do correlations stabilize?Journal of Research in Personality, 47(5), 609–612. https://doi.org/10.1016/j.jrp.2013.05.009
54.
SeymourP. H. K. (1977). Conceptual encoding and locus of the Stroop effect. Quarterly Journal of Experimental Psychology, 29(2), 245–265. https://doi.org/10.1080/14640747708400601
55.
ShroutP. E.FleissJ. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428. https://doi.org/10.1037/0033-2909.86.2.420
56.
SimonJ. R.SmallA. M. (1969). Processing auditory information: Interference from an irrelevant cue. Journal of Applied Psychology, 53(5), 433–435. https://doi.org/10.1037/h0028034
57.
StirlingN. (1979). Stroop Interference: An input and an output phenomenon. Quarterly Journal of Experimental Psychology, 31(1), 121–132. https://doi.org/10.1080/14640747908400712
58.
StoetG. (2010). PsyToolkit: A software package for programming psychological experiments using Linux. Behavior Research Methods, 42(4), 1096–1104. https://doi.org/10.3758/BRM.42.4.1096
59.
StoetG. (2017). PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 44(1), 24–31. https://doi.org/10.1177/0098628316677643
60.
SzűcsD.SoltészF.WhiteS. (2009). Motor conflict in Stroop tasks: Direct evidence from single-trial electro-myography and electro-encephalography. NeuroImage, 47(4), 1960–1973. https://doi.org/10.1016/j.neuroimage.2009.05.048
61.
TipperS. P. (2001). Does negative priming reflect inhibitory mechanisms? A review and integration of conflicting views. Quarterly Journal of Experimental Psychology Section A, 54(2), 321–343. https://doi.org/10.1080/713755969
62.
TsalY.BenoniH. (2010). Diluting the burden of load: Perceptual load effects are simply dilution effects. Journal of Experimental Psychology: Human Perception and Performance, 36(6), 1645–1656. https://doi.org/10.1037/a0018172
63.
Van Der LindenW. J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31(2), 181–204. https://doi.org/10.3102/10769986031002181
64.
Van VeenV.CarterC. S. (2005). Separating semantic conflict and response conflict in the Stroop task: A functional MRI study. NeuroImage, 27(3), 497–504. https://doi.org/10.1016/j.neuroimage.2005.04.042
65.
Von BastianC. C.BlaisC.BrewerG. A.GyurkovicsM.HedgeC.KałamałaP.MeierM. E.OberauerK.Rey-MermetA.RouderJ. N.SouzaA. S.BartschL. M.ConwayA. R. A.DraheimC.EngleR. W.FriedmanN. P.FrischkornG. T.GustavsonD. E.KochI.WiemersE. A. (2020). Advancing the understanding of individual differences in attentional control: Theoretical, methodological, and analytical considerations. PsyArXiv. https://doi.org/10.31234/osf.io/x3b9k
66.
VoormannA.MillerJ. (2024). Sequential effects on reaction time distributions: Commonalities and differences across paradigms. Journal of Cognition, 7(1), 68. https://doi.org/10.5334/joc.395
67.
WagenmakersE.-J.BrownS. (2007). On the linear relation between the mean and the standard deviation of a response time distribution. Psychological Review, 114(3), 830–841. https://doi.org/10.1037/0033-295X.114.3.830
68.
WeissmanD. H.EgnerT.HawksZ.LinkJ. (2015). The congruency sequence effect emerges when the distracter precedes the target. Acta Psychologica, 156, 8–21. https://doi.org/10.1016/j.actpsy.2015.01.003
69.
WendtM.KieselA.GeringswaldF.PurmannS.FischerR. (2014). Attentional adjustment to conflict strength: Evidence from the effects of manipulating flanker-target SOA on response times and prestimulus pupil size. Experimental Psychology, 61(1), 55–67. https://doi.org/10.1027/1618-3169/a000227
YuanY.LiG.RenH.ChenW. (2020). Caffeine effect on cognitive function during a Stroop task: fNIRS study. Neural Plasticity, 2020, 1–8. https://doi.org/10.1155/2020/8833134
72.
ZhangH.KornblumS. (1998). The effects of stimulus–response mapping and irrelevant stimulus–response and stimulus–stimulus overlap in four-choice Stroop tasks with single-carrier stimuli. Journal of Experimental Psychology: Human Perception and Performance, 24(1), 3–19. https://doi.org/10.1037/0096-1523.24.1.3
73.
ZhangH. (Harry)ZhangJ.KornblumS. (1999). A parallel distributed processing model of stimulus–stimulus and stimulus–response compatibility. Cognitive Psychology, 38(3), 386–432. https://doi.org/10.1006/cogp.1998.0703
74.
ZorowitzS.NivY. (2023). Improving the reliability of cognitive task measures: A narrative review. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging, 8(8), 789–797. https://doi.org/10.1016/j.bpsc.2023.02.004
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
