Adaptive assessment of the capacity of cognitive control

Abstract

Cognitive control, although it has limited capacity, serves an essential role in supporting a broad range of cognitive functions. The backward masking majority function task (MFT-M) is a validated behavioural method for measuring the capacity of cognitive control (CCC), but the administration is lengthy. We tested the relative efficiency of administering the MFT-M using an adaptive method based on the principles of computerised adaptive testing (CAT). Participants were 40 healthy young adults aged 18–26. Scores on the adaptive version were highly correlated with scores based on the original approach to administration and showed high test–retest reliability. In addition, compared with the original task of 864 trials (about 86 min), less than 216 trials (20 min) were required in the adaptive version. The results suggest that CAT is a valid and more efficient method for assessing CCC than the MFT-M. This study provides an example of adaptive trial selection in task administration, an approach that can advance the methodology of behavioural science.

Keywords

Cognitive control capacity of cognitive control backward masking majority function task computerised adaptive testing

Get full access to this article

View all access options for this article.

References

Botvinick

M. M.

Braver

T. S.

Barch

D. M.

Carter

C. S.

Cohen

J. D.

(2001). Conflict monitoring and cognitive control. Psychological Review, 108(3), 624–652. https://doi.org/10.1037/0033-295x.108.3.624

Brodke

D. J.

Hung

Bozic

K. J.

(2016). Item response theory and computerized adaptive testing for orthopaedic outcomes measures. Journal of the American Academy of Orthopaedic Surgeons, 24(11), 750–754. https://doi.org/10.5435/jaaos-d-15-00420

Chang

H. H.

(2015). Psychometrics behind computerized adaptive testing. Psychometrika, 80(1), 1–20. https://doi.org/10.1007/s11336-014-9401-5

Chang

H. H.

Ying

(1996). A global information approach to computerized adaptive testing. Applied Psychological Measurement, 20(3), 213–229. https://doi.org/10.1177/014662169602000303

Chen

Qiu

Zhang

Fan

(2020). Accessing the development and heritability of the capacity of cognitive control. Neuropsychologia, 139, Article 107361. https://doi.org/10.1016/j.neuropsychologia.2020.107361

Chen

Spagna

Kim

T. H.

Chen

Fan

(2019). Testing a cognitive control model of human intelligence. Scientific Reports, 9(1), Article 2898. https://doi.org/10.1038/s41598-019-39685-2

Cheng

(2009). When cognitive diagnosis meets computerized adaptive testing: CD-CAT. Psychometrika, 74(4), 619–632. https://doi.org/10.1007/s11336-009-9123-2

Choi

S. W.

Grady

M. W.

Dodd

B. G.

(2010). A new stopping rule for computerized adaptive testing. Educational and Psychological Measurement, 71(1), 37–53. https://doi.org/10.1177/0013164410387338

Egner

(2007). Congruency sequence effects and cognitive control. Cognitive, Affective, & Behavioral Neuroscience, 7(4), 380–390. https://doi.org/10.3758/cabn.7.4.380

10.

Egner

Hirsch

(2005). Cognitive control mechanisms resolve conflict through cortical amplification of task-relevant information. Nature Neuroscience, 8(12), 1784–1790. https://doi.org/10.1038/nn1594

11.

Embretson

S. E.

Reise

S. P.

(2000). Item response theory for psychologists. Lawrence Erlbaum.

12.

Fan

(2014). An information theory account of cognitive control. Frontiers in Human Neuroscience, 8, Article 680. https://doi.org/10.3389/fnhum.2014.00680

13.

Fan

Guise

K. G.

Liu

Wang

(2008). Searching for the majority: Algorithms of voluntary control. PLOS ONE, 3(10), Article e3522. https://doi.org/10.1371/journal.pone.0003522

14.

Fan

Van Dam

N. T.

Liu

Wang

Tang

C. Y.

Hof

P. R.

(2014). Quantitative characterization of functional anatomical contributions to cognitive control under uncertainty. Journal of Cognitive Neuroscience, 26(7), 1490–1506. https://doi.org/10.1162/jocn_a_00554

15.

Fliege

Becker

Walter

O. B.

Bjorner

J. B.

Rose

K. M.

(2005). Development of a computer-adaptive test for depression (D-CAT). Quality of Life Research, 14(10), 2277–2291. https://www.jstor.org/stable/4039962

16.

Fraser

Cockcroft

(2019). Working with memory: Computerized, adaptive working memory training for adolescents living with HIV. Child Neuropsychology, 26(5), 612–634. https://doi.org/10.1080/09297049.2019.1676407

17.

Gibbons

R. D.

Weiss

D. J.

Kupfer

D. J.

Frank

Fagiolini

Grochocinski

V. J.

Bhaumik

D. K.

Stover

Bock

Immekus

J. C.

(2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59(4), 361–368. https://doi.org/10.1176/ps.2008.59.4.361

18.

Gibbons

R. D.

Weiss

D. J.

Pilkonis

P. A.

Frank

Moore

Kim

J. B.

Kupfer

D. J.

(2012). Development of a computerized adaptive test for depression. Archives of General Psychiatry, 69(11), Article 1104. https://doi.org/10.1001/archgenpsychiatry.2012.14

19.

Gibbons

R. D.

Weiss

D. J.

Pilkonis

P. A.

Frank

Moore

Kim

J. B.

Kupfer

D. J.

(2014). Development of the CAT-ANX: A computerized adaptive test for anxiety. American Journal of Psychiatry, 171(2), 187–194. https://doi.org/10.1176/appi.ajp.2013.13020178

20.

Chen

Wang

Qiu

Fan

Guan

Luo

(2019). Reduced capacity of cognitive control in older adults with mild cognitive impairment. Journal of Alzheimer’s Disease, 71(1), 185–200. https://doi.org/10.3233/jad-181006

21.

Hick

W. E.

(1952). On the rate of gain of information. Quarterly Journal of Experimental Psychology, 4(1), 11–26. https://doi.org/10.1080/17470215208416600

22.

Hyman

(1953). Stimulus information as a determinant of reaction time. Journal of Experimental Psychology, 45(3), 188–196. https://doi.org/10.1037/h0056940

23.

Kingsbury

G. G.

Zara

A. R.

(1989). Procedures for selecting items for computerized adaptive tests. Applied Measurement in Education, 2(4), 359–375. https://doi.org/10.1207/s15324818ame0204_6

24.

Klingberg

(2010). Training and plasticity of working memory. Trends in Cognitive Sciences, 14(7), 317–324. https://doi.org/10.1016/j.tics.2010.05.002

25.

Klingberg

Fernell

Olesen

P. J.

Johnson

Gustafsson

Dahlstrom

Gillberg

C. J.

Forssberg

Westerberg

(2005). Computerized training of working memory in children with ADHD—A randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44(2), 177–186. https://doi.org/10.1097/00004583-200502000-00010

26.

Loosli

S. V.

Buschkuehl

Perrig

W. J.

Jaeggi

S. M.

(2012). Working memory training improves reading processes in typically developing children. Child Neuropsychology, 18(1), 62–78. https://doi.org/10.1080/09297049.2011.575772

27.

Luo

Kim

Dickison

(2018). Projection-based stopping rules for computerized adaptive testing in licensure testing. Applied Psychological Measurement, 42(4), 275–290. https://doi.org/10.1177/0146621617726790

28.

Mackie

M. A.

Dam

N. T. V.

Fan

(2013). Cognitive control and attentional functions. Brain and Cognition, 82(3), 301–312. https://doi.org/10.1016/j.bandc.2013.05.004

29.

Meijer

R. R.

Nering

M. L.

(1999). Computerized adaptive testing: Overview and introduction. Applied Psychological Measurement, 23(3), 187–194. https://doi.org/10.1177/01466219922031310

30.

Menendez

M. E.

Bot

A. G. J.

Hageman

M. G. J. S.

Neuhaus

Mudgal

C. S.

Ring

(2013). Computerized adaptive testing of psychological factors. The Journal of Bone & Joint Surgery, 95(20), Article e149. https://doi.org/10.2106/jbjs.l.01614

31.

Peirce

Gray

J. R.

Simpson

MacAskill

Höchenberger

Sogo

Kastman

Lindeløv

J. K.

(2019). PsychoPy2: Experiments in behavior made easy. Behavior Research Methods, 51, 195–203. https://doi.org/10.3758/s13428-018-01193-y

32.

Primi

Morsanyi

Chiesi

Donati

M. A.

Hamilton

(2016). The development and testing of a new version of the cognitive reflection test applying item response theory (IRT). Journal of Behavioral Decision Making, 29(5), 453–469. https://doi.org/10.1002/bdm.1883

33.

Şenel

Kutlu

. (2018). Comparison of two test methods for VIS: Paper-pencil test and CAT. European Journal of Special Needs Education, 33(5), 631–645. https://doi.org/10.1080/08856257.2017.1391014

34.

Smits

Cuijpers

van Straten

(2011). Applying computerized adaptive testing to the CES-D Scale: A simulation study. Psychiatry Research, 188(1), 147–155. https://doi.org/10.1016/j.psychres.2010.12.001

35.

Stark

Chernyshenko

O. S.

Drasgow

White

L. A.

(2012). Adaptive testing with multidimensional pairwise preference items. Organizational Research Methods, 15(3), 463–487. https://doi.org/10.1177/1094428112444611

36.

Stochl

Böhnke

J. R.

Pickett

K. E.

Croudace

T. J.

(2016). An evaluation of computerized adaptive testing for general psychological distress: Combining GHQ-12 and Affectometer-2 in an item bank for public mental health research. BMC Medical Research Methodology, 16(1), Article 58. https://doi.org/10.1186/s12874-016-0158-7

37.

Stroop

J. R.

(1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 18(6), 643–662. https://doi.org/10.1037/h0054651

38.

Wainer

(Ed.). (2000). Computerized adaptive testing: A primer (2nd ed.). Lawrence Erlbaum.

39.

Wang

Weiss

D. J.

Shang

(2018). Variable-length stopping rules for multidimensional computerized adaptive testing. Psychometrika, 84, 749–771. https://doi.org/10.1007/s11336-018-9644-7

40.

Wang

Liu

Fan

(2011). Cognitive control in majority search: A computational modeling approach. Frontiers in Human Neuroscience, 5, Article 16. https://doi.org/10.3389/fnhum.2011.00016

41.

Weiss

D. J.

(2011). Better data from better measurements using computerized adaptive testing. Journal of Methods and Measurement in the Social Sciences, 2(1), Article 1. https://doi.org/10.2458/v2i1.12351

42.

Weiss

D. J.

Kingsbury

G. G.

(1984). Application of computerized adaptive testing of educational problems. Journal of Educational Measurement, 21(4), 361–375. https://doi.org/10.1111/j.1745-3984.1984.tb01040.x

43.

Veerkamp

W. J. J.

Berger

M. P. F.

(1997). Some new item selection criteria for adaptive testing. Journal of Educational and Behavioral Statistics, 22(2), 203–226. https://doi.org/10.2307/1165378

44.

Dufford

A. J.

Egan

L. J.

Mackie

M. A.

Chen

Yuan

Chen

Liu

Hof

P. R.

Fan

(2018). Hick-Hyman Law is mediated by the cognitive control network in the brain. Cerebral Cortex, 28(7), 2267–2282. https://doi.org/10.1093/cercor/bhx127

45.

Dufford

A. J.

Mackie

M. A.

Egan

L. J.

Fan

(2016). The capacity of cognitive control estimated from a perceptual decision making task. Scientific Reports, 6(1), Article 34025. https://doi.org/10.1038/srep34025

46.

Wang

Spagna

Yang

Yuan

Gao

Hof

P. R.

Fan

(2019a). Anterior insular cortex is a bottleneck of cognitive control. NeuroImage, 195, 490–504. https://doi.org/10.1016/j.neuroimage.2019.02.042

47.

Cai

(2019b). A computerized adaptive testing advancing the measurement of subjective well-being. Journal of Pacific Rim Psychology, 13, Article e6. https://doi.org/10.1017/prp.2019.6

48.

Jin

Huang

Zhou

Zhang

(2020). Development of computerized adaptive testing for emotion regulation. Frontiers in Psychology, 11, Article 56135. https://doi.org/10.3389/fpsyg.2020.561358

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.08 MB