The Rosenberg Self-Esteem Scale is the most frequently used measure of self-esteem in the social sciences. These items are often administered with a different number of response options, but it is unclear how the number of response options impacts the psychometric properties of this measure. Across three experiments (Ns = 739, 2,358, and 1,461), we evaluated how different response options of the Rosenberg influenced (a) coefficient alpha estimates, (b) distributions of scores, and (c) associations with criterion-related variables. Observed coefficient alpha estimates were lowest for a 2-point format compared with response formats with more options. However, supplemental analyses using ordinal alpha pointed to similar estimates across conditions. Using four or more response options better approximated a normal distribution for observed summary scores. We found no consistent evidence that criterion-related correlations increased with more response options. Collectively, these results suggest that the Rosenberg should be administered with at least four response options and we favor a 5-point Likert-type response format.
AdelsonJ. L.McCoachD. B. (2010). Measuring the mathematical attitudes of elementary students: The effects of a 4-point or 5-point Likert-type scale. Educational and Psychological Measurement, 70, 796–807.
2.
AguinisH.PierceC. A.CulpepperS. A. (2009). Scale coarseness as a methodological artifact: Correcting correlation coefficients attenuated from using coarse scales. Organizational Research Methods, 12(4), 623–652.
3.
AlessandriG.VecchioneM.EisenbergN.ŁagunaM. (2015). On the factor structure of the Rosenberg (1965) general self-esteem scale. Psychological Assessment, 27(2), 621–635.
4.
AshtonM. C.LeeK. (2007). Empirical, theoretical, and practical advantages of the HEXACO model of personality structure. Personality and Social Psychology Review, 11, 150–166.
5.
AustF.DiedenhofenB.UllrichS.MuschJ. (2013). Seriousness checks are useful to improve data validity in online research. Behavior Research Methods, 45, 527–535.
6.
BaumeisterR. F.CampbellJ. D.KruegerJ. I.VohsK. D. (2003). Does high self-esteem cause better performance, interpersonal success, happiness, or healthier lifestyles?Psychological Science in the Public Interest, 4, 1–44.
7.
BlascovichJ.TomakaJ. (1991). Measures of self-esteem. In RobinsonJ. P.ShaverP. R.WrightsmanL. S. (Eds.), Measures of personality and social psychological attitudes (pp. 115–160). Academic Press.
8.
BollenK. A.BarbK. H. (1981). Pearson’s r and coarsely categorized measures. American Sociological Review, 46, 232–239.
9.
ChalmersR. P. (2018). On misconceptions and the limited usefulness of ordinal alpha. Educational and Psychological Measurement, 78(6), 1056–1071.
10.
CliftonJ. D. (2020). Managing validity versus reliability trade-offs in scale-building decisions. Psychological Methods, 25, 259–270.
11.
CochranW. G. (1954). The combination of estimates from different experiments. Biometrics, 10, 101–129.
12.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.
13.
CohenP.CohenJ.AikenL. S.WestS. G. (1999). The problem of units and the circumstance for POMP. Multivariate Behavioral Research, 34, 315–346.
14.
CoxA.CourrégéS. C.FederA. H.WeedN. C. (2017). Effects of augmenting response options of the MMPI-2-RF: An extension of previous findings. Cogent Psychology, 4, 1323988.
15.
DawesJ. (2008). Do data characteristics change according to the number of scale points used? An experiment using 5-point, 7-point and 10-point scales. International Journal of Market Research, 50, 61–104.
16.
DiedenhofenB.MuschJ. (2016). cocron: A web interface and R package for the statistical comparison of Cronbach’s Alpha Coefficients. International Journal of Internet Science, 11, 51–60.
17.
DienerE. D.EmmonsR. A.LarsenR. J.GriffinS. (1985). The satisfaction with life scale. Journal of Personality Assessment, 49, 71–75.
18.
DonnellanM. B.AckermanR. A.BrecheenC. (2016). Extending structural analyses of the Rosenberg self-esteem scale to consider criterion-related validity: Can composite self-esteem scores be good enough?Journal of Personality Assessment, 98, 169–177.
19.
DonnellanM. B.TrzesniewskiK. H.RobinsR. W. (2015). Measures of self-esteem. In BoyleG. J.SaklofskeD. H.MatthewsG. (Eds.), Measures of personality and social psychological constructs (pp. 131–157). Academic Press.
20.
FinnJ. A.Ben-PorathY. S.TellegenA. (2015). Dichotomous versus polytomous response options in psychopathology assessment: Method or meaningful variance?Psychological Assessment, 27, 184–193.
21.
FloraD. B. (2020). Your coefficient alpha is probably wrong, but which coefficient omega is right? A tutorial on using R to obtain better reliability estimates. Advances in Methods and Practices in Psychological Science, 3, 484–501.
22.
GadermannA. M.GuhnM.ZumboB. D. (2012). Estimating ordinal reliability for Likert-type and ordinal item response data: A conceptual, empirical, and practical guide. Practical Assessment, Research, and Evaluation, 17(1), Article 3.
23.
GnambsT.ScharlA.SchroedersU. (2018). The structure of the Rosenberg Self-Esteem Scale: A cross-cultural meta-analysis. Zeitschrift für Psychologie, 226(1), 14–29.
24.
Gray-LittleB.WilliamsV. S.HancockT. D. (1997). An item response theory analysis of the Rosenberg Self-Esteem Scale. Personality and Social Psychology Bulletin, 23, 443–451.
25.
HeathertonT. F.WylandC. L. (2003). Assessing self-esteem. In LopezS. J.SnyderC. R. (Eds.), Positive psychological assessment: A handbook of models and measures (pp. 219–233). American Psychological Association.
26.
HuangC.DongN. (2012). Factor structures of the Rosenberg Self-Esteem Scale: A meta-analysis of pattern matrices. European Journal of Psychological Assessment, 28(2), 132–138.
27.
JacobyJ.MatellM. S. (1971). Three-point Likert scales are good enough. Journal of Marketing Research, 8, 495–500.
28.
LanceC. E.ButtsM. M.MichelsL. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say?Organizational Research Methods, 9, 202–220.
29.
LeeK.AshtonM. C. (2018). Psychometric properties of the HEXACO-100. Assessment, 25, 543–556.
30.
LeungS. O. (2011). A comparison of psychometric properties and normality in 4-, 5-, 6-, and 11-point Likert scales. Journal of Social Service Research, 37, 412–421.
31.
LiuH. Y.WengL. J. (2009). An effect size index for comparing two independent alpha coefficients. British Journal of Mathematical and Statistical Psychology, 62(2), 385–400.
32.
LucasR. E.DienerE.SuhE. (1996). Discriminant validity of well-being measures. Journal of Personality and Social Psychology, 71, 616–628.
33.
MatellM. S.JacobyJ. (1971). Is there an optimal number of alternatives for Likert scale items? Study I: Reliability and validity. Educational and Psychological Measurement, 31, 657–674.
34.
McNeishD. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412–433.
35.
MüssigM.KubiakJ.EgloffB. (2021). The agony of choice: Acceptance, efficiency, and psychometric properties of questionnaires with different numbers of response options. Assessment. Advance online publication. https://doi.org/10.1177/10731911211029379
36.
OrthU.RobinsR. W. (2022). Is high self-esteem beneficial? Revisiting a classic question. American Psychologist, 77(1), 5–17.
37.
PatrickC. J. (2010). Operationalizing the triarchic conceptualization of psychopathy: Preliminary description of brief scales for assessment of boldness, meanness, and disinhibition [Unpublished manual]. Department of Psychology, Florida State University.
38.
PatrickC. J.DrislaneL. E. (2015). Triarchic model of psychopathy: Origins, operationalizations, and observed linkages with personality and general psychopathology. Journal of Personality, 83, 627–643.
39.
PatrickC. J.FowlesD. C.KruegerR. F. (2009). Triarchic conceptualization of psychopathy: Developmental origins of disinhibition, boldness, and meanness. Development and Psychopathology, 21, 913–938.
40.
PeglerA. J.GreggA. P.HartC. M. (2019) The Rosenberg and the rest: Meta-research on the measurement of self-esteem in personality and social psychology (2004-2015). PsyArXiv. https://doi.org/10.31234/osf.io/aps9m
41.
RevelleW.CondonD. M. (2019). Reliability from α to ω: A tutorial. Psychological Assessment, 31, 1395–1411.
42.
RhemtullaM.Brosseau-LiardP. É.SavaleiV. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354–373.
43.
RobinsR. W.HendinH. M.TrzesniewskiK. H. (2001). Measuring global self-esteem: Construct validation of a single-item measure and the Rosenberg Self-Esteem Scale. Personality and Social Psychology Bulletin, 27, 151–161.
44.
RobinsR. W.TracyJ. L.TrzesniewskiK.PotterJ.GoslingS. D. (2001). Personality correlates of self-esteem. Journal of Research in Personality, 35, 463–482.
45.
RosenbergM. (1989). Society and the adolescent self-image (Rev. ed.). Wesleyan University Press.
46.
RuchenskyJ. R.DonnellanM. B. (2017). Integrating the HEXACO model with the Triarchic conceptualization of psychopathy. Personality and Individual Differences, 119, 129–133.
47.
SavaleiV.ReiseS. P. (2019). Don’t forget the model in your model-based reliability coefficients: A reply to McNeish (2018). Collabra: Psychology, 5, Article 36. https://doi.org/10.1525/collabra.247
48.
SchmittD. P.AllikJ. (2005). Simultaneous administration of the Rosenberg Self-Esteem Scale in 53 nations: exploring the universal and culture-specific features of global self-esteem. Journal of Personality and Social Psychology, 89(4), 623–642.
49.
SchmidtF. L.HunterJ. E. (1996). Measurement error in psychological research: Lessons from 26 research scenarios. Psychological Methods, 1(2), 199–223.
50.
SchmittN. (1996). Uses and abuses of coefficient alpha. Psychological Assessment, 8(4), 350–353.
51.
SimmsL. J.ZelaznyK.WilliamsT. F.BernsteinL. (2019). Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychological Assessment, 31, 557–556.
52.
SotoC. J.JohnO. P. (2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113, 117–143.
53.
StreinerD. L. (2003). Starting at the beginning: An introduction to coefficient alpha and internal consistency. Journal of Personality Assessment, 80, 99–103.
54.
Zeigler-HillV. (2013). The importance of self-esteem. Psychology Press.
55.
ZumboB. D.GadermannA. M.ZeisserC. (2007). Ordinal versions of coefficients alpha and theta for Likert rating scales. Journal of Modern Applied Statistical Methods, 6(1), 21–29.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
