Do Rating and Task Measures of Control Abilities Assess the Same Thing?

Abstract

The ability to control one’s thoughts and actions is broadly associated with health and success, so it is unsurprising that measuring self-control abilities is a common goal across many areas of psychology. Puzzlingly, however, different measures of control––questionnaire ratings and cognitive tasks––show only weak relationships to each other. We review evidence that this discrepancy is not just a result of poor reliability or validity of ratings or tasks. Rather, ratings and tasks seem to assess different aspects of control, distinguishable along six main dimensions. To improve the psychological science surrounding self-control, it will be important for future work to investigate the relative importance of these dimensions to the dissociations between self-control measures, and for researchers to explain which aspects of control they are studying and why they have focused on those aspects of control when one or both types of measures are deployed.

Keywords

cognitive control executive function self-regulation self-control impulsivity

Get full access to this article

View all access options for this article.

References

Barkley

R. A.

Fischer

(2011). Predicting impairment in major life activities and occupational functioning in hyperactive children as adults: Self-reported executive function (EF) deficits versus EF tests. Developmental Neuropsychology, 36(2), 137–161. https://doi.org/10.1080/87565641.2010.549877

Broadbent

D. E.

Cooper

P. F.

FitzGerald

Parkes

K. R.

(1982). The Cognitive Failures Questionnaire (CFQ) and its correlates. British Journal of Clinical Psychology, 21(1), 1–16. https://doi.org/10.1111/j.2044-8260.1982.tb01421.x

Cyders

M. A.

Coskunpinar

(2011). Measurement of constructs using self-report and behavioral lab tasks: Is there overlap in nomothetic span and construct representation for impulsivity? Clinical Psychology Review, 31(6), 965–982. https://doi.org/10.1016/j.cpr.2011.06.001

Dang

King

K. M.

Inzlicht

(2020). Why are self-report and behavioral measures weakly correlated? Trends in Cognitive Sciences, 24(4), 267–269. https://doi.org/10.1016/j.tics.2020.01.007

de Ridder

D. T. D.

Lensvelt-Mulders

Finkenauer

Stok

F. M.

Baumeister

R. F.

(2012). Taking stock of self-control: A meta-analysis of how trait self-control relates to a wide range of behaviors. Personality and Social Psychology Review, 16(1), 76–99. https://doi.org/10.1177/1088868311418749

Diamond

(2013). Executive functions. Annual Review of Psychology, 64, 135–168. https://doi.org/10.1146/annurev-psych-113011-143750

Duckworth

A. L.

Kern

M. L.

(2011). A meta-analysis of the convergent validity of self-control measures. Journal of Research in Personality, 45(3), 259–268. https://doi.org/10.1016/j.jrp.2011.02.004

Duckworth

A. L.

Quinn

P. D.

(2009). Development and validation of the Short Grit Scale (Grit–S). Journal of Personality Assessment, 91(2), 166–174. https://doi.org/10.1080/00223890802634290

Eisenberg

I. W.

Bissett

P. G.

Enkavi

A. Z.

MacKinnon

D. P.

Marsch

L. A.

Poldrack

R. A.

(2019). Uncovering the structure of self-regulation through data-driven ontology discovery. Nature Communications, 10(1), Article 2319. https://doi.org/10.1038/s41467-019-10301-1

10.

Ellingson

J. M.

Corley

Hewitt

J. K.

Friedman

N. P.

(2019). A prospective study of alcohol involvement and the dual-systems model of adolescent risk-taking during late adolescence and emerging adulthood. Addiction, 114(4), 653–661. https://doi.org/10.1111/add.14489

11.

Ellis

L. K.

Rothbart

M. K.

(2001). Revision of the Early Adolescent Temperament Questionnaire. https://research.bowdoin.edu/rothbart-temperament-questionnaires/files/2016/09/lesa-ellis-srcd-poster-reprint.pdf

12.

Enkavi

A. Z.

Eisenberg

I. W.

Bissett

P. G.

Mazza

G. L.

MacKinnon

D. P.

Marsch

L. A.

Poldrack

R. A.

(2019). Large-scale analysis of test–retest reliabilities of self-regulation measures. Proceedings of the National Academy of Sciences, USA, 116(12), 5472–5477. https://doi.org/10.1073/pnas.1818430116

13.

Friedman

N. P.

Hatoum

A. S.

Gustavson

D. E.

Corley

R. P.

Hewitt

J. K.

Young

S. E.

(2020). Executive functions and impulsivity are genetically distinct and independently predict psychopathology: Results from two adult twin studies. Clinical Psychological Science, 8(3), 519–538. https://doi.org/10.1177/2167702619898814

14.

Friedman

N. P.

Miyake

(2004). The relations among inhibition and interference control functions: A latent-variable analysis. Journal of Experimental Psychology: General, 133(1), 101–135. https://doi.org/10.1037/0096-3445.133.1.101

15.

Friedman

N. P.

Miyake

(2017). Unity and diversity of executive functions: Individual differences as a window on cognitive structure. Cortex, 86, 186–204. https://doi.org/10.1016/j.cortex.2016.04.023

16.

Friedman

N. P.

Miyake

Altamirano

L. J.

Corley

R. P.

Young

S. E.

Rhea

S. A.

Hewitt

J. K.

(2016). Stability and change in executive function abilities from late adolescence to early adulthood: A longitudinal twin study. Developmental Psychology, 52(2), 326–340. https://doi.org/10.1037/dev0000075

17.

Galla

B. M.

Duckworth

A. L.

(2015). More than resisting temptation: Beneficial habits mediate the relationship between self-control and positive life outcomes. Journal of Personality and Social Psychology, 109(3), 508–525. https://doi.org/10.1037/pspp0000026

18.

Gioia

G. A.

Isquith

P. K.

Guy

S. C.

Kenworthy

(2000). TEST REVIEW: Behavior Rating Inventory of Executive Function. Child Neuropsychology, 6(3), 235–238. https://doi.org/10.1076/chin.6.3.235.3152

19.

Grund

Carstens

C.-A.

(2019). Self-control motivationally reconsidered: “Acting” self-controlled is different to “being good” at self-control. Motivation and Emotion, 43(1), 63–81. https://doi.org/10.1007/s11031-018-9721-3

20.

Gustavson

D. E.

Panizzon

M. S.

Elman

J. A.

Franz

C. E.

Reynolds

C. A.

Jacobson

K. C.

Friedman

N. P.

Xian

Toomey

Lyons

M. J.

Kremen

W. S.

(2018). Stability of genetic and environmental influences on executive functions in midlife. Psychology and Aging, 33(2), 219–231. https://doi.org/10.1037/pag0000230

21.

Gustavson

D. E.

Stallings

M. C.

Corley

R. P.

Miyake

Hewitt

J. K.

Friedman

N. P.

(2017). Executive functions and substance use: Relations in late adolescence and early adulthood. Journal of Abnormal Psychology, 126(2), 257–270. https://doi.org/10.1037/abn0000250

22.

Hedge

Powell

Sumner

(2018). The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences. Behavior Research Methods, 50(3), 1166–1186. https://doi.org/10.3758/s13428-017-0935-1

23.

Hofmann

Baumeister

R. F.

Förster

Vohs

K. D.

(2012). Everyday temptations: An experience sampling study of desire, conflict, and self-control. Journal of Personality and Social Psychology, 102(6), 1318–1335. https://doi.org/10.1037/a0026545

24.

Kamradt

J. M.

Ullsperger

J. M.

Nikolas

M. A.

(2014). Executive function assessment and adult attention-deficit/hyperactivity disorder: Tasks versus ratings on the Barkley Deficits in Executive Functioning Scale. Psychological Assessment, 26(4), 1095–1105. https://doi.org/10.1037/pas0000006

25.

Lynam

D. R.

Whiteside

S. P.

Smith

G. T.

Cyders

M. A.

(2006). The UPPS-P: Assessing five personality pathways to impulsive behavior [Unpublished manuscript]. Department of Psychological Sciences, Purdue University, West Lafayette.

26.

Malanchini

Engelhardt

L. E.

Grotzinger

A. D.

Harden

K. P.

Tucker-Drob

E. M.

(2019). “Same but different”: Associations between multiple aspects of self-regulation, cognition, and academic abilities. Journal of Personality and Social Psychology, 117(6), 1164–1188. https://doi.org/10.1037/pspp0000224

27.

Mischel

Baker

(1975). Cognitive appraisals and transformations in delay behavior. Journal of Personality and Social Psychology, 31(2), 254–261. https://doi.org/10.1037/h0076272

28.

Patton

J. H.

Stanford

M. S.

Barratt

E. S.

(1995). Factor structure of the Barratt Impulsiveness Scale. Journal of Clinical Psychology, 51(6), 768–774. https://doi.org/10.1002/1097-4679(199511)51:6<768::AID-JCLP2270510607>3.0.CO;2-1

29.

Rey-Mermet

Gade

Oberauer

(2018). Should we stop thinking about inhibition? Searching for individual and age differences in inhibition ability. Journal of Experimental Psychology: Learning, Memory, and Cognition, 44(4), 501–526. https://doi.org/10.1037/xlm0000450

30.

Sharma

Markon

K. E.

Clark

L. A.

(2014). Toward a theory of distinct types of “impulsive” behaviors: A meta-analysis of self-report and behavioral measures. Psychological Bulletin, 140(2), 374–408. https://doi.org/10.1037/a0034418

31.

Snyder

H. R.

Friedman

N. P.

Hankin

B. L.

(2021). Associations between task performance and self-report measures of cognitive control: Shared versus distinct abilities. Assessment, 28(4), 1080–1096. https://doi.org/10.1177/1073191120965694

32.

Snyder

H. R.

Miyake

Hankin

B. L.

(2015). Advancing understanding of executive function impairments and psychopathology: Bridging the gap between clinical and cognitive approaches. Frontiers in Psychology, 6, Article 328. https://doi.org/10.3389/fpsyg.2015.00328

33.

Soto

C. J.

John

O. P.

(2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113(1), 117–143. https://doi.org/10.1037/pspp0000096

34.

Soto

E. F.

Kofler

M. J.

Singh

L. J.

Wells

E. L.

Irwin

L. N.

Groves

N. B.

Miller

C. E.

(2020). Executive functioning rating scales: Ecologically valid or construct invalid? Neuropsychology, 34(6), 605–619. https://doi.org/10.1037/neu0000681

35.

Steel

(2010). Arousal, avoidant and decisional procrastinators: Do they exist? Personality and Individual Differences, 48(8), 926–934. https://doi.org/10.1016/j.paid.2010.02.025

36.

Stunkard

A. J.

Messick

(1985). The Three-Factor Eating Questionnaire to measure dietary restraint, disinhibition and hunger. Journal of Psychosomatic Research, 29(1), 71–83. https://doi.org/10.1016/0022-3999(85)90010-8

37.

Tangney

J. P.

Baumeister

R. F.

Boone

A. L.

(2004). High self-control predicts good adjustment, less pathology, better grades, and interpersonal success. Journal of Personality, 72(2), 271–324. https://doi.org/10.1111/j.0022-3506.2004.00263.x

38.

Toplak

M. E.

West

R. F.

Stanovich

K. E.

(2013). Practitioner Review: Do performance-based measures and ratings of executive function assess the same construct? Journal of Child Psychology and Psychiatry, 54(2), 131–143. https://doi.org/10.1111/jcpp.12001

39.

Verdejo-Garcia

Tiego

Kakoschke

Moskovsky

Voigt

Anderson

Koutoulogenis

Lubman

D. I.

Bellgrove

M. A.

(2021). A unified online test battery for cognitive impulsivity reveals relationships with real-world impulsive behaviours. Nature Human Behaviour, 5(11), 1562–1577. https://doi.org/10.1038/s41562-021-01127-3

40.

Vohs

K. D.

Piquero

A. R.

(2021). Self-control at 220 miles per hour: Steering and braking to achieve optimal outcomes during adolescence. Current Directions in Psychological Science, 30(4), 351–357. https://doi.org/10.1177/09637214211014509

41.

Wegner

D. M.

Zanakos

(1994). Chronic thought suppression. Journal of Personality, 62(4), 615–640. https://doi.org/10.1111/j.1467-6494.1994.tb00311.x

42.

Wennerhold

Friese

(2020). Why self-report measures of self-control and inhibition tasks do not substantially correlate. Collabra: Psychology, 6(1), Article 9. https://doi.org/10.1525/collabra.276

43.

White

R. E.

Carlson

S. M.

(2016). What would Batman do? Self-distancing improves executive function in young children. Developmental Science, 19(3), 419–426. https://doi.org/10.1111/desc.12314

44.

Zelazo

P. D.

Carlson

S. M.

(2012). Hot and cool executive function in childhood and adolescence: Development and plasticity. Child Development Perspectives, 6(4), 354–360. https://doi.org/10.1111/j.1750-8606.2012.00246.x