Cognitive control, although it has limited capacity, serves an essential role in supporting a broad range of cognitive functions. The backward masking majority function task (MFT-M) is a validated behavioural method for measuring the capacity of cognitive control (CCC), but the administration is lengthy. We tested the relative efficiency of administering the MFT-M using an adaptive method based on the principles of computerised adaptive testing (CAT). Participants were 40 healthy young adults aged 18–26. Scores on the adaptive version were highly correlated with scores based on the original approach to administration and showed high test–retest reliability. In addition, compared with the original task of 864 trials (about 86 min), less than 216 trials (20 min) were required in the adaptive version. The results suggest that CAT is a valid and more efficient method for assessing CCC than the MFT-M. This study provides an example of adaptive trial selection in task administration, an approach that can advance the methodology of behavioural science.
BotvinickM. M.BraverT. S.BarchD. M.CarterC. S.CohenJ. D. (2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 624–652. https://doi.org/10.1037/0033-295x.108.3.624
2.
BrodkeD. J.HungM.BozicK. J. (2016). Item response theory and computerized adaptive testing for orthopaedic outcomes measures. Journal of the American Academy of Orthopaedic Surgeons, 24(11), 750–754. https://doi.org/10.5435/jaaos-d-15-00420
ChangH. H.YingZ. (1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213–229. https://doi.org/10.1177/014662169602000303
5.
ChenY.ChenC.WuT.QiuB.ZhangW.FanJ. (2020). Accessing the development and heritability of the capacity of cognitive control. Neuropsychologia, 139, Article 107361. https://doi.org/10.1016/j.neuropsychologia.2020.107361
6.
ChenY.SpagnaA.WuT.KimT. H.WuQ.ChenC.WuY.FanJ. (2019). Testing a cognitive control model of human intelligence. Scientific Reports, 9(1), Article 2898. https://doi.org/10.1038/s41598-019-39685-2
ChoiS. W.GradyM. W.DoddB. G. (2010). A new stopping rule for computerized adaptive testing. Educational and Psychological Measurement, 71(1), 37–53. https://doi.org/10.1177/0013164410387338
FanJ.Van DamN. T.GuX.LiuX.WangH.TangC. Y.HofP. R. (2014). Quantitative characterization of functional anatomical contributions to cognitive control under uncertainty. Journal of Cognitive Neuroscience, 26(7), 1490–1506. https://doi.org/10.1162/jocn_a_00554
15.
FliegeH.BeckerJ.WalterO. B.BjornerJ. B.RoseK. M. (2005). Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research, 14(10), 2277–2291. https://www.jstor.org/stable/4039962
16.
FraserS.CockcroftK. (2019). Working with memory: Computerized, adaptive working memory training for adolescents living with HIV. Child Neuropsychology, 26(5), 612–634. https://doi.org/10.1080/09297049.2019.1676407
17.
GibbonsR. D.WeissD. J.KupferD. J.FrankE.FagioliniA.GrochocinskiV. J.BhaumikD. K.StoverA.BockD.ImmekusJ. C. (2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59(4), 361–368. https://doi.org/10.1176/ps.2008.59.4.361
18.
GibbonsR. D.WeissD. J.PilkonisP. A.FrankE.MooreT.KimJ. B.KupferD. J. (2012). Development of a computerized adaptive test for depression. Archives of General Psychiatry, 69(11), Article 1104. https://doi.org/10.1001/archgenpsychiatry.2012.14
19.
GibbonsR. D.WeissD. J.PilkonisP. A.FrankE.MooreT.KimJ. B.KupferD. J. (2014). Development of the CAT-ANX: A computerized adaptive test for anxiety. American Journal of Psychiatry, 171(2), 187–194. https://doi.org/10.1176/appi.ajp.2013.13020178
20.
HeH.XuP.WuT.ChenY.WangJ.QiuY.FanJ.GuanQ.LuoY. (2019). Reduced capacity of cognitive control in older adults with mild cognitive impairment. Journal of Alzheimer’s Disease, 71(1), 185–200. https://doi.org/10.3233/jad-181006
HymanR. (1953). Stimulus information as a determinant of reaction time. Journal of Experimental Psychology, 45(3), 188–196. https://doi.org/10.1037/h0056940
23.
KingsburyG. G.ZaraA. R. (1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2(4), 359–375. https://doi.org/10.1207/s15324818ame0204_6
KlingbergT.FernellE.OlesenP. J.JohnsonM.GustafssonP.DahlstromK.GillbergC. J.ForssbergH.WesterbergH. (2005). Computerized training of working memory in children with ADHD—A randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44(2), 177–186. https://doi.org/10.1097/00004583-200502000-00010
26.
LoosliS. V.BuschkuehlM.PerrigW. J.JaeggiS. M. (2012). Working memory training improves reading processes in typically developing children. Child Neuropsychology, 18(1), 62–78. https://doi.org/10.1080/09297049.2011.575772
27.
LuoX.KimD.DickisonP. (2018). Projection-based stopping rules for computerized adaptive testing in licensure testing. Applied Psychological Measurement, 42(4), 275–290. https://doi.org/10.1177/0146621617726790
MeijerR. R.NeringM. L. (1999). Computerized adaptive testing: Overview and introduction. Applied Psychological Measurement, 23(3), 187–194. https://doi.org/10.1177/01466219922031310
30.
MenendezM. E.BotA. G. J.HagemanM. G. J. S.NeuhausV.MudgalC. S.RingD. (2013). Computerized adaptive testing of psychological factors. The Journal of Bone & Joint Surgery, 95(20), Article e149. https://doi.org/10.2106/jbjs.l.01614
31.
PeirceJ.GrayJ. R.SimpsonS.MacAskillM.HöchenbergerR.SogoH.KastmanE.LindeløvJ. K. (2019). PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51, 195–203. https://doi.org/10.3758/s13428-018-01193-y
32.
PrimiC.MorsanyiK.ChiesiF.DonatiM. A.HamiltonJ. (2016). The development and testing of a new version of the cognitive reflection test applying item response theory (IRT). Journal of Behavioral Decision Making, 29(5), 453–469. https://doi.org/10.1002/bdm.1883
33.
ŞenelS.KutluÖ. (2018). Comparison of two test methods for VIS: Paper-pencil test and CAT. European Journal of Special Needs Education, 33(5), 631–645. https://doi.org/10.1080/08856257.2017.1391014
34.
SmitsN.CuijpersP.van StratenA. (2011). Applying computerized adaptive testing to the CES-D Scale: A simulation study. Psychiatry Research, 188(1), 147–155. https://doi.org/10.1016/j.psychres.2010.12.001
35.
StarkS.ChernyshenkoO. S.DrasgowF.WhiteL. A. (2012). Adaptive testing with multidimensional pairwise preference items. Organizational Research Methods, 15(3), 463–487. https://doi.org/10.1177/1094428112444611
36.
StochlJ.BöhnkeJ. R.PickettK. E.CroudaceT. J. (2016). An evaluation of computerized adaptive testing for general psychological distress: Combining GHQ-12 and Affectometer-2 in an item bank for public mental health research. BMC Medical Research Methodology, 16(1), Article 58. https://doi.org/10.1186/s12874-016-0158-7
37.
StroopJ. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. https://doi.org/10.1037/h0054651
38.
WainerH. (Ed.). (2000). Computerized adaptive testing: A primer (2nd ed.). Lawrence Erlbaum.
WangH.LiuX.FanJ. (2011). Cognitive control in majority search: A computational modeling approach. Frontiers in Human Neuroscience, 5, Article 16. https://doi.org/10.3389/fnhum.2011.00016
41.
WeissD. J. (2011). Better data from better measurements using computerized adaptive testing. Journal of Methods and Measurement in the Social Sciences, 2(1), Article 1. https://doi.org/10.2458/v2i1.12351
42.
WeissD. J.KingsburyG. G. (1984). Application of computerized adaptive testing of educational problems. Journal of Educational Measurement, 21(4), 361–375. https://doi.org/10.1111/j.1745-3984.1984.tb01040.x
43.
VeerkampW. J. J.BergerM. P. F. (1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22(2), 203–226. https://doi.org/10.2307/1165378
44.
WuT.DuffordA. J.EganL. J.MackieM. A.ChenC.YuanC.ChenC.LiX.LiuX.HofP. R.FanJ. (2018). Hick-Hyman Law is mediated by the cognitive control network in the brain. Cerebral Cortex, 28(7), 2267–2282. https://doi.org/10.1093/cercor/bhx127
45.
WuT.DuffordA. J.MackieM. A.EganL. J.FanJ. (2016). The capacity of cognitive control estimated from a perceptual decision making task. Scientific Reports, 6(1), Article 34025. https://doi.org/10.1038/srep34025
46.
WuT.WangX.WuQ.SpagnaA.YangJ.YuanC.WuY.GaoZ.HofP. R.FanJ. (2019a). Anterior insular cortex is a bottleneck of cognitive control. NeuroImage, 195, 490–504. https://doi.org/10.1016/j.neuroimage.2019.02.042
47.
WuY.CaiY.TuD. (2019b). A computerized adaptive testing advancing the measurement of subjective well-being. Journal of Pacific Rim Psychology, 13, Article e6. https://doi.org/10.1017/prp.2019.6
48.
XuL.JinR.HuangF.ZhouY.LiZ.ZhangM. (2020). Development of computerized adaptive testing for emotion regulation. Frontiers in Psychology, 11, Article 56135. https://doi.org/10.3389/fpsyg.2020.561358
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
