Reliability and Validity of the Self-Rating Scale as a Measure of Self-Criticism

Abstract

The Self-Rating Scale (SRS; Hooley et al., 2010), a widely used measure of self-criticism in self-injury research, did not utilize conventional test development methods and has limited psychometric data. We examined the internal consistency, test–retest reliability, and convergent and discriminant validity of the SRS. Participants were 295 psychology undergraduate students. The SRS demonstrated good internal consistency (α = .93), adequate test–retest reliability (r = .76), and satisfactory convergent validity with other measures of self-criticism. Convergent validity was also adequate for expected dimensions of perfectionism (socially prescribed, self-oriented, concerns about mistakes, and doubts about actions), depressive symptoms, and negative and positive affect. The SRS demonstrated adequate discriminant validity with expected constructs of perfectionism (other-oriented, personal standards, and organizational perfectionism). Although the SRS appears to be a psychometrically sound measure of self-criticism, high correlations with depression and perfectionism raise questions regarding the overlap of these constructs.

Keywords

self-criticism perfectionism negative affect depressive symptoms test–retest reliability internal consistency validity

Get full access to this article

View all access options for this article.

References

Allen

K. J.

Fox

K. R.

Schatten

H. T.

Hooley

J. M.

(2019). Frequency of nonsuicidal self-injury is associated with impulsive decision-making during criticism. Psychiatry Research, 271, 68–75. https://doi.org/10.1016/j.psychres.2018.11.022

Baetens

Claes

Hasking

Smits

Grietens

Onghena

Martin

(2015). The relationship between parental expressed emotions and non-suicidal self-injury: The mediating roles of self-criticism and depression. Journal of Child and Family Studies, 24(2), 491–498. https://doi.org/10.1007/s10826-013-9861-8

Bagby

R. M.

Parker

J. D.

Joffe

R. T.

Buis

(1994). Reconstruction and validation of the Depressive Experiences Questionnaire. Assessment, 1(1), 59–68. https://doi.org/10.1177/1073191194001001009

Beck

A. T.

Steer

R. A.

Brown

G. K.

(1996). Manual for the Beck Depression Inventory-II. The Psychological Corporation.

Beck

A. T.

Weissman

Lester

Trexler

(1974). The measurement of pessimism: The Hopelessness Scale. Journal of Consulting and Clinical Psychology, 42(6), 861–865. https://doi.org/10.1037/h0037562

Bernstein

E. M.

Putnam

F. W.

(1986). Development, reliability, and validity of a dissociation scale. Journal of Nervous and Mental Disease, 174(12), 727–735. https://doi.org/10.1097/00005053-198612000-00004

Blatt

S. J.

D’Afflitti

J. P.

Quinlan

D. M.

(1976). Experiences of depression in normal young adults. Journal of Abnormal Psychology, 85(4), 383. https://doi.org/10.1037/0021-843x.85.4.383

Blankstein

K. R.

Dunkley

D. M.

(2002). Evaluative concerns, self-critical, and personal standards perfectionism: A structural equation modeling. In Flett

G. L.

Hewitt

P. L.

(Eds.), Perfectionism: Theory, research, and treatment (pp. 285–316). American Psychological Association.

Burke

T. A.

Fox

Kautz

M. M.

Rodriguez-Seijas

Bettis

A. H.

Alloy

L. B.

(2021a). Self-critical and self-punishment cognitions differentiate those with and without a history of nonsuicidal self-injury: An ecological momentary assessment study. Behavior Therapy, 52(3), 686–697. https://doi.org/10.1016/j.beth.2020.08.006

10.

Burke

T. A.

Fox

Kautz

M. M.

Siegel

D. M.

Kleiman

Alloy

L. B.

(2021b). Real-time monitoring of the associations between self-critical and self-punishment cognitions and nonsuicidal self-injury. Behaviour Research and Therapy, 137, 103775. https://doi.org/10.1016/j.brat.2020.103775

11.

Burke

T. A.

Fox

Zelkowitz

R. L.

Smith

D. M.

Alloy

L. B.

Hooley

J. M.

Cole

D. A.

(2019). Does nonsuicidal self-injury prospectively predict change in depression and self-criticism? Cognitive Therapy and Research, 43(2), 345–353. https://doi.org/10.1007/s10608-018-9984-8

12.

Carver

C. S.

Ganellen

R. J.

(1983). Depression and components of self-punitiveness: High standards, self-criticism, and overgeneralization. Journal of Abnormal Psychology, 92(3), 330–337. https://doi.org/10.1037/0021-843x.92.3.330

13.

Chui

Zilcha-Mano

Dinger

Barrett

M. S.

Barber

J. P.

(2016). Dependency and self-criticism in treatments for depression. Journal of Counseling Psychology, 63(4), 452. https://doi.org/10.1037/cou0000142

14.

Clark

L. A.

Watson

(1991). Tripartite model of anxiety and depression: Psychometric evidence and taxonomic implications. Journal of Abnormal Psychology, 100(3), 316–336. https://doi.org/10.1037/0021-843X.100.3.316

15.

Costa

P. T.

McCrae

R. R.

(1992). Revised NEO Personality Inventory (NEO–PI–R) and NEO Five Factor Inventory (NEO–FFI): Professional manual. Psychological Assessment Resources.

16.

Craig

A. R.

Franklin

J. A.

Andrews

(1984). A scale to measure locus of control of behaviour. British Journal of Medical Psychology, 57(2), 173–180. https://doi.org/10.1111/j.2044-8341.1984.tb01597.x

17.

Duek

Shahar

Osher

Kofman

(2018). Role of self-criticism in reward and punishment probabilistic learning. Personality and Individual Differences, 134, 155–163. https://doi.org/10.1016/j.paid.2018.06.007

18.

Dunkley

D. M.

Blankstein

K. R.

(2000). Self-critical perfectionism, coping, hassles, and current distress: A structural equation modeling approach. Cognitive Therapy and Research, 24(6), 713–730. https://doi.org/10.1023/A:1005543529245

19.

Dunkley

D. M.

Zuroff

D. C.

Blankstein

K. R.

(2006). Specific perfectionism components versus self-criticism in predicting maladjustment. Personality and Individual Differences, 40(4), 665–676. https://doi.org/10.1016/j.paid.2005.08.008

20.

Enns

M. W.

Cox

B. J.

(2002). The nature and assessment of perfectionism: A critical analysis. In Flett

G. L.

Hewitt

P. L.

(Eds.), Perfectionism: Theory, research, and treatment (pp. 33–62). American Psychological Association.

21.

Frost

R. O.

Heimberg

R. G.

Holt

C. S.

Mattia

J. I.

Neubauer

A. L.

(1993). A comparison of two measures of perfectionism. Personality and Individual Differences, 14(1), 119–126. https://doi.org/10.1016/0191-8869(93)90181-2

22.

Frost

R. O.

Marten

Lahart

Rosenblate

(1990). The dimensions of perfectionism. Cognitive Therapy and Research, 14(5), 449–468. https://doi.org/10.1007/BF01172967

23.

Gilbert

Clarke

Hempel

Miles

J. N. V.

Irons

(2004). Criticizing and reassuring oneself: An exploration of forms, styles and reasons in female students. British Journal of Clinical Psychology, 43(1), 31–50. https://doi.org/10.1348/014466504772812959

24.

Glassman

L. H.

Weierich

M. R.

Hooley

J. M.

Deliberto

T. L.

Nock

M. K.

(2007). Child maltreatment, non-suicidal self-injury, and the mediating role of self-criticism. Behaviour Research and Therapy, 45(10), 2483–2490. https://doi.org/10.1016/j.brat.2007.04.002

25.

Hewitt

P. L.

Flett

G. L.

(1991). Perfectionism in the self and social contexts: Conceptualization, assessment, and association with psychopathology. Journal of Personality and Social Psychology, 60(3), 456–470. https://doi.org/10.1037/0022-3514.60.3.456

26.

Hewitt

P. L.

Flett

G. L.

Turnbull-Donovan

Mikail

S. F.

(1991). The Multidimensional Perfectionism Scale: Reliability, validity, and psychometric properties in psychiatric samples. Psychological Assessment: A Journal of Consulting and Clinical Psychology, 3(3), 464–468. https://doi.org/10.1037/1040-3590.3.3.464

27.

Hill

R. W.

McIntire

Bacharach

V. R.

(1997). Perfectionism and the big five factors. Journal of Social Behavior and Personality, 12(1), 257. https://doi.org/10.1017/bec.2015.3

28.

Hooley

J. M.

D. T.

Slater

J. A.

Lockshin

(2002). Pain insensitivity and self-harming behavior. Paper presented at the Annual Meeting of the Society for Research in Psychopathology, San Francisco, CA, United States, September 26-29.

29.

Hooley

J. M.

D. T.

Slater

J. A.

Lockshin

(2010). Pain perception and nonsuicidal self-injury: A laboratory investigation. Personality Disorders: Theory, Research, and Treatment, 1(3), 170–179. https://doi.org/10.1037/a0020106

30.

IBM Corp. (2013, Released). IBM SPSS statistics for windows (Version 22.0). IBM Corp.

31.

Ishiyama

F. I.

Munson

P. A.

(1993). Development and validation of a self-critical cognition scale. Psychological Reports, 72(1), 147–154. https://doi.org/10.2466/pr0.1993.72.1.147

32.

Kamholz

B. W.

Hayes

A. M.

Carver

C. S.

Gulliver

S. B.

Perlman

C. A.

(2006). Identification and evaluation of cognitive affect-regulation strategies: Development of a self-report measure. Cognitive Therapy and Research, 30(2), 227–262. https://doi.org/10.1007/s10608-006-9013-1

33.

Kaplan

R. M.

Saccuzzo

D. P.

(2017). Psychological testing: Principles, applications, and issues. Nelson Education.

34.

Lear

M. K.

Wilkowski

B. M.

Pepper

C. M.

(2019). A daily diary investigation of the defective self model among college students with recent self-injury. Behavior Therapy, 50(5), 1002–1012. https://doi.org/10.1016/j.beth.2019.03.005

35.

Longe

Maratos

F. A.

Gilbert

Evans

Volker

Rockliff

Rippon

(2010). Having a word with yourself: Neural correlates of self-criticism and self-reassurance. NeuroImage, 49(2), 1849–1856. https://doi.org/10.1016/j.neuroimage.2009.09.019

36.

Luyten

Sabbe

Blatt

S. J.

Meganck

Jansen

De Grave

Maes

Corveleyn

(2007). Dependency and self-criticism: Relationship with major depressive disorder, severity of depression, and clinical presentation. Depression and Anxiety, 24(8), 586–596. https://doi.org/10.1002/da.20272

37.

Manfredi

Caselli

Pescini

Rossi

Rebecchi

Ruggiero

G. M.

Sassaroli

(2016). Parental criticism, self-criticism and their relation to depressive mood: An exploratory study among a non-clinical population. Research in Psychotherapy: Psychopathology, Process and Outcome, 19(1), 41–48. https://doi.org/10.4081/ripppo.2016.178

38.

McIntyre

Smith

Rimes

K. A.

(2018). The role of self-criticism in common mental health difficulties in students: A systematic review of prospective studies. Mental Health & Prevention, 10, 13–27. https://doi.org/10.1016/j.mhp.2018.02.003

39.

Meyer

I. H.

(2003). Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: Conceptual issues and research evidence. Psychological Bulletin, 129(5), 674–697. https://doi.org/10.1037/0033-2909.129.5.674

40.

Neff

K. D.

(2003). The development and validation of a scale to measure self-compassion. Self and Identity, 2(3), 223–250. https://doi.org/10.1080/15298860309027

41.

O’Connor

R. C.

(2007). The relations between perfectionism and suicidality: A systematic review. Suicide and Life-Threatening Behavior, 37(6), 698–714. https://doi.org/10.1521/suli.2007.37.6.698

42.

Parker

Manicavasagar

Crawford

J. O.

Tully

Gladstone

(2006). Assessing personality traits associated with depression: the utility of a tiered model. Psychological Medicine, 36(8), 1131–1139. https://doi.org/10.1017/s0033291706007562

43.

Porter

A. C.

Zelkowitz

R. L.

Gist

D. C.

Cole

D. A.

(2019). Self-evaluation and depressive symptoms: A latent variable analysis of self-esteem, shame-proneness, and self-criticism. Journal of Psychopathology and Behavioral Assessment, 41(2), 257–270. https://doi.org/10.1007/s10862-019-09734-1

44.

Rose

A. V.

Rimes

K. A.

(2018). Self-criticism self-report measures: Systematic review. Psychology and Psychotherapy, 91(4), 450–489. https://doi.org/10.1111/papt.12171

45.

Sedikides

Luke

(2008). On when self-enhancement and self-criticism function adaptively and maladaptively. In Chang

E. C.

(Eds.), Self-criticism and self-enhancement (pp. 181–198). American Psychological Association.

46.

Smart

L. M.

Peters

J. R.

Baer

R. A.

(2015). Development and validation of a measure of self-critical rumination. Assessment, 23, 321–332. https://doi.org/10.1177/1073191115573300

47.

St. Germain

S. A. S.

Hooley

J. M

. (2012). Direct and indirect forms of non-suicidal self-injury: Evidence for a distinction. Psychiatry Research, 197(1–2), 78–84. https://doi.org/10.1016/j.psychres.2011.12.050

48.

Thew

G. R.

Gregory

J. D.

Roberts

Rimes

K. A.

(2017). Self-critical thinking and overgeneralization in depression and eating disorders: An experimental study. Behavioural and Cognitive Psychotherapy, 45(5), 510–523. https://doi.org/10.1017/S1352465817000327

49.

Thompson

Zuroff

D. C.

(2004). The Levels of Self-Criticism Scale: Comparative self-criticism and internalized self-criticism. Personality and Individual Differences, 36(2), 419–430. https://doi.org/10.1016/S0191-8869(03)00106-5

50.

Watson

(2004). Stability versus change, dependability versus error: Issues in the assessment of personality over time. Journal of Research in Personality, 38, 319–350. https://doi.org/10.1016/j.jrp.2004.03.001

51.

Watson

Clark

L. A.

(1994). The PANAS-X: Manual for the positive and negative affect schedule-expanded form. The University of Iowa.

52.

Wedig

M. M.

Nock

M. K.

(2007). Parental expressed emotion and adolescent self-injury. Journal of the American Academy of Child & Adolescent Psychiatry, 46(9), 1171–1178. https://doi.org/10.1097/chi.0b013e3180ca9aaf

53.

Weissman

A. N.

Beck

A. T.

(1978). Development and validation of the Dysfunctional Attitude Scale: A preliminary investigation. Paper presented at the Annual Meeting of the American Educational Research Association, Toronto, Canada.

54.

Wheeler

H. A.

Blankstein

K. R.

Antony

M. M.

McCabe

R. E.

Bieling

P. J.

(2011). Perfectionism in anxiety and depression: Comparisons across disorders, relations with symptom severity, and role of comorbidity. International Journal of Cognitive Therapy, 4(1), 66–91. https://doi.org/10.1521/ijct.2011.4.1.66

55.

Wiggins

J. S.

(1973). Personality and prediction: Principles of personality assessment. Addison-Wesley.

56.

Zuroff

D. C.

Quinlan

D. M.

Blatt

S. J.

(1990). Psychometric properties of the Depressive Experiences Questionnaire in a college population. Journal of Personality Assessment, 55(1–2), 65–72. https://doi.org/10.1080/00223891.1990.9674047

57.

Zuroff

D. C.

Sadikaj

Kelly

A. C.

Leybman

M. J.

(2016). Conceptualizing and measuring self-criticism as both a personality trait and a personality state. Journal of Personality Assessment, 98(1), 14–21. https://doi.org/10.1080/00223891.2015.1044604