There is consensus that the 10 items of the Rosenberg Self-Esteem Scale (RSES) reflect wording effects resulting from positively and negatively keyed items. The present study examined the effects of cognitive abilities on the factor structure of the RSES with a novel, nonparametric latent variable technique called local structural equation models. In a nationally representative German large-scale assessment including 12,437 students competing measurement models for the RSES were compared: a bifactor model with a common factor and a specific factor for all negatively worded items had an optimal fit. Local structural equation models showed that the unidimensionality of the scale increased with higher levels of reading competence and reasoning, while the proportion of variance attributed to the negatively keyed items declined. Wording effects on the factor structure of the RSES seem to represent a response style artifact associated with cognitive abilities.
AlessandriG.VecchioneM.EisenbergN.ŁagunaM. (2015). On the factor structure of the Rosenberg (1965) General Self-Esteem Scale. Psychological Assessment, 27, 621-635. doi:10.1037/pas0000073
2.
BeauducelA.HerzbergP. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling, 13, 186-203. doi:10.1207/s15328007sem1302_2
3.
BlossfeldH.-P.RoßbachH.-Gvon MauriceJ. (2011). Education as a lifelong process. Zeitschrift für Erziehungswissenschaft, 14(Suppl. 2), 19-34.
4.
BrunnerM.NagyG.WilhelmO. (2012). A tutorial on hierarchically structured constructs. Journal of Personality, 80, 796-846. doi:10.1111/j.1467-6494.2011.00749.x
5.
ChaoR. C. L.VidacovichC.GreenK. E. (2017). Rasch analysis of the Rosenberg Self-Esteem Scale with African Americans. Psychological Assessment, 29, 329-342. doi:10.1037/pas0000347
6.
CorderyJ. L.SevastosP. P. (1993). Responses to the original and revised job diagnostic survey: Is education a factor in responses to negatively worded items?Journal of Applied Psychology, 78, 141-143. doi:10.1037/0021-9010.78.1.141
7.
CorwynR. F. (2000). The factor structure of global self-esteem among adolescents and adults. Journal of Research in Personality, 34, 357-379. doi:10.1006/jrpe.2000.2291
8.
DisethÅ.MelandE.BreidablikH. J. (2014). Self-beliefs among students: Grade level and gender differences in self-esteem, self-efficacy and implicit theories of intelligence. Learning and Individual Differences, 35, 1-8. doi:10.1016/j.lindif.2014.06.003
9.
DiStefanoC.MotlR. W. (2006). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling, 13, 440-464. doi:10.1207/s15328007sem1303_6
10.
DiStefanoC.MotlR. W. (2009). Self-esteem and method effects associated with negatively worded items: Investigating factorial invariance by sex. Structural Equation Modeling, 16, 134-146. doi:10.1080/10705510802565403
11.
DonnellanM. B.AckermanR. A.BrecheenC. (2016). Extending structural analyses of the Rosenberg Self-Esteem Scale to consider criterion-related validity: Can composite self-esteem scores be good enough?Journal of Personality Assessment, 98, 169-177. doi:10.1080/00223891.2015.1058268
12.
DonnellanM. B.TrzesniewskiK. H.RobinsR. W. (2011). Self-esteem: Enduring issues and controversies. In Chamorro-PremuzicT.von StummS.FurnhamA. (Eds.), The Wiley-Blackwell handbook of individual differences (pp. 718-746). New York, NY: Wiley-Blackwell.
13.
DunbarM.FordG.HuntK.DerG. (2000). Question wording effects in the assessment of global self-esteem. European Journal of Psychological Assessment, 16, 13-19. doi:10.1027//1015-5759.16.1.13
14.
DunnL. M.DunnD. M. (2004). Peabody Picture Vocabulary Test (PPVT) (German version). Göttingen, Germany: Hogrefe.
15.
EidM.GeiserC.KochT.HeeneM. (2017). Anomalous results in G-factor models: Explanations and alternatives. Psychological Methods, 22, 541-562. doi:10.1037/met0000083
16.
FreedmanD. A. (2006). On the so-called “Huber sandwich estimator” and “robust standard errors.”American Statistician, 60, 299-302. doi:10.1198/000313006X152207
17.
FranckE.De RaedtR.BarbezC.RosseelY. (2008). Psychometric properties of the Dutch Rosenberg Self-Esteem Scale. Psychologica Belgica, 48, 25-35. doi:10.5334/pb-48-1-25
18.
GanaK.SaadaY.BaillyN.JoulainM.HervéC.AlaphilippeD. (2013). Longitudinal factorial invariance of the Rosenberg Self-Esteem Scale: Determining the nature of method effects due to item wording. Journal of Research in Personality, 47, 406-416. doi:10.1016/j.jrp.2013.03.011
19.
GasserT.GerviniD.MolinariL. (2004). Kernel estimation, shape-invariant modeling and structural analysis. In HauspieR.CameronN.MolinariL. (Eds.), Methods in human growth research (pp. 179-204). Cambridge, England: Cambridge University Press.
20.
GnambsT.BuntinsK. (2017). The measurement of variability and change in life satisfaction: A comparison of single-item and multi-item instruments. European Journal of Psychological Assessment, 33, 224-238. doi:10.1027/1015-5759/a000414
21.
GnambsT.ScharlA.SchroedersU. (2018). The structure of the Rosenberg Self-Esteem Scale: A cross-cultural meta-analysis. Zeitschrift für Psychologie.
22.
GreenS. B.YangY. (2009). Reliability of summed item scores using structural equation modeling: An alternative to coefficient alpha. Psychometrika, 74, 155-167. doi:10.1007/s11336-008-9099-3
23.
GreenbergerE.ChenC.DmitrievaJ.FarruggiaS. P. (2003). Item wording and the dimensionality of the Rosenberg Self-Esteem Scale: Do they matter?Personality and Individual Differences, 35, 1241-1254. doi:10.1016/S0191-8869(02)00331-8
24.
GuH.WenZ.FanX. (2017). Examining and controlling for wording effect in a self-report measure: A Monte Carlo simulation study. Structural Equation Modeling, 24, 545-555. doi:10.1080/10705511.2017.1286228
25.
HaberkornK.PohlS.HardtK.WiegandE. (2012). NEPS technical report for reading: Scaling results of starting Cohort 4 in ninth grade (NEPS Working Paper No. 16). Bamberg, Germany: Otto-Friedrich-Universität, Nationales Bildungspanel.
26.
HildebrandtA.LüdtkeO.RobitzschA.SommerC.WilhelmO. (2016). Exploring factor model parameters across continuous variables with local structural equation models. Multivariate Behavioral Research, 51, 257-258. doi:10.1080/00273171.2016.1142856
27.
HildebrandtA.SommerW.HerzmannG.WilhelmO. (2010). Structural invariance and age-related performance differences in face cognition. Psychology and Aging, 25, 794-810. doi:10.1037/a0019774
28.
HildebrandtA.WilhelmO.HerzmannG.SommerW. (2013). Face and object cognition across adult age. Psychology and Aging, 28, 243-248. doi:10.1037/a0031490
29.
HildebrandtA.WilhelmO.RobitzschA. (2009). Complementary and competing factor analytic approaches for the investigation of measurement invariance. Review of Psychology, 16, 87-102.
30.
HoranP. M.DiStefanoC.MotlR. W. (2003). Wording effects in self-esteem scales: Methodological artifact or response style?Structural Equation Modeling, 10, 435-455. doi:10.1207/s15328007sem1003_6
31.
HülürG.WilhelmO.RobitzschA. (2011). Intelligence differentiation in early childhood. Journal of Individual Differences, 32, 170-179. doi:10.1027/1614-0001/a000049
32.
JohnsonT.KulesaP.ChoY. I.ShavittS. (2005). The relation between culture and response styles: Evidence from 19 countries. Journal of Cross-Cultural Psychology, 36, 264-277. doi:10.1177/0022022104272905
33.
LechnerC. M.RammstedtB. (2015). Cognitive ability, acquiescence, and the structure of personality in a sample of older adults. Psychological Assessment, 27, 1301-1311. doi:10.1037/pas0000151
34.
LindwallM.BarkoukisV.GranoC.LucidiF.RaudseppL.LiukkonenJ.Thøgersen-NtoumaniC. (2012). Method effects: The problem with negatively versus positively keyed items. Journal of Personality Assessment, 94, 196-204. doi:10.1080/00223891.2011.645936
35.
MacCallumR. C.ZhangS.PreacherK. J.RuckerD. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7, 19-40. doi:10.1037/1082-989X.7.1.19
36.
MarshH. W. (1996). Positive and negative global self-esteem: A substantively meaningful distinction or artifactors?Journal of Personality and Social Psychology, 70, 810-819. doi:10.1037/0022-3514.70.4.810
37.
MarshH. W.NagengastB.MorinA. J. (2013). Measurement invariance of big-five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and la dolce vita effects. Developmental Psychology, 49, 1194-1218. doi:10.1037/a0026913
38.
MarshH. W.ScalasL. F.NagengastB. (2010). Longitudinal tests of competing factor structures for the Rosenberg Self-Esteem Scale: Traits, ephemeral artifacts, and stable response styles. Psychological Assessment, 22, 366-381. doi:10.1037/a0019225
39.
McNeishD. (2018). Thanks coefficient alpha, we’ll take it from here. Psychological Methods, 23(3), 412. doi:10.1037/met0000144
40.
MeisenbergG.WilliamsA. (2008). Are acquiescent and extreme response styles related to low intelligence and education?Personality and Individual Differences, 44, 1539-1550. doi:10.1016/j.paid.2008.01.010
41.
MichaelidesM. P.KoutsogiorgiC.PanayiotouG. (2016). Method effects on an adaptation of the Rosenberg Self-Esteem Scale in Greek and the role of personality traits. Journal of Personality Assessment, 98, 178-188. doi:10.1080/00223891.2015.1089248
42.
MichaelidesM. P.ZengerM.KoutsogiorgiC.BrählerE.Stöbel-RichterY.BerthH. (2016). Personality correlates and gender invariance of wording effects in the German version of the Rosenberg Self-Esteem Scale. Personality and Individual Differences, 97, 13-18. doi:10.1016/j.paid.2016.03.011
43.
MotlR. W.DiStefanoC. (2002). Longitudinal invariance of self-esteem and method effects associated with negatively worded items. Structural Equation Modeling, 9, 562-578. doi:10.1207/S15328007SEM0904_6
44.
NosekB. A.AlterG.BanksG. C.BorsboomD.BowmanS. D.BrecklerS. J.. . . YarkoniT. (2015). Promoting an open research culture. Science, 348, 1420-1422. doi:10.1126/science.aab2374
45.
OwensT. J. (1994). Two dimensions of self-esteem: Reciprocal effects of positive self-worth and self-deprecation on adolescent problems. American Sociological Review, 59, 391-407. doi:10.2307/2095940
46.
PreacherK. J.RuckerD. D.MacCallumR. C.NicewanderW. A. (2005). Use of the extreme groups approach: A critical reexamination and new recommendations. Psychological Methods, 10, 178-192. doi:10.1037/1082-989X.10.2.178
47.
PullmannH.AllikJ. (2000). The Rosenberg Self-Esteem Scale: Its dimensionality, stability and personality correlates in Estonian. Personality and Individual Differences, 28, 701-715. doi:10.1016/S0191-8869(99)00132-4
48.
QuiltyL. C.OakmanJ. M.RiskoE. (2006). Correlates of the Rosenberg Self-Esteem Scale method effects. Structural Equation Modeling, 13, 99-117. doi:10.1207/s15328007sem1301_5
49.
R Core Team. (2017). R: A language and environment for statistical computing (Computer software), Vienna, Austria. Retrieved from https://www.R-project.org/
50.
RammstedtB.FarmerR. F. (2013). The impact of acquiescence on the evaluation of personality structure. Psychological Assessment, 25, 1137-1145. doi:10.1037/a0033323
51.
RammstedtB.KemperC. J.BorgI. (2013). Correcting Big Five personality measurements for acquiescence: An 18-country cross-cultural study. European Journal of Personality, 27, 71-81. doi:10.1002/per.1894
52.
RaschG. (1960). Probabilistic models for some intelligence and attainment tests. Copenhagen, Denmark: Danish Institute for Educational Research.
53.
ReiseS. P. (2012): The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47, 667-696. doi:10.1080/00273171.2012.715555
54.
ReiseS. P.KimD. S.MansolfM.WidamanK. F. (2016). Is the bifactor model a better model or is it just better at modeling implausible responses? Application of iteratively reweighted least squares to the Rosenberg Self-Esteem Scale. Multivariate Behavioral Research, 51, 818-838. doi:10.1080/00273171.2016.1243461
55.
ReiseS. P.MooreT. M.HavilandM. G. (2010). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores. Journal of Personality Assessment, 92, 544-559. doi:10.1080/00223891.2010.496477
56.
RhemtullaM.Brosseau-LiardP. É.SavaleiV. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17, 354-373. doi:10.1037/a0029315
RodriguezA.ReiseS. P.HavilandM. G. (2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21, 137-150. doi:10.1037/met0000045
59.
RosenbergM. (1965). Society and the adolescent self-image. Princeton, NJ: Princeton University Press.
60.
RosseelY. (2012). lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48, 1-36. doi:10.18637/jss.v048.i02
61.
RothM.DeckerO.HerzbergP. Y.BrählerE. (2008). Dimensionality and norms of the Rosenberg Self-Esteem Scale in a German general population sample. European Journal of Psychological Assessment, 24, 190-197. doi:10.1027/1015-5759.24.3.190
62.
RuckerD. D.McShaneB. B.PreacherK. J. (2015). A researcher’s guide to regression, discretization, and median splits of continuous variables. Journal of Consumer Psychology, 25, 666-678. doi:10.1016/j.jcps.2015.04.004
63.
SalernoL.IngogliaS.CocoG. L. (2017). Competing factor structures of the Rosenberg Self-Esteem Scale (RSES) and its measurement invariance across clinical and non-clinical samples. Personality and Individual Differences, 113, 13-19. doi:10.1016/j.paid.2017.02.063
64.
Schermelleh-EngelK.MoosbruggerH.MüllerH. (2003). Evaluating the fit of structural equation models: Test of significance and descriptive goodness-of-fit measures. Methods of Psychological Research Online, 8, 23-74.
65.
SchmittD. P.AllikJ. (2005). Simultaneous administration of the Rosenberg Self-Esteem Scale in 53 nations: Exploring the universal and culture-specific features of global self-esteem. Journal of Personality and Social Psychology, 89, 623-643. doi:10.1037/0022-3514.89.4.623
66.
SchroedersU.SchipolowskiS.WilhelmO. (2015). Age-related changes in the mean and covariance structure of fluid and crystallized intelligence in childhood and adolescence. Intelligence, 48, 15-29. doi:10.1016/j.intell.2014.10.006
67.
SchulzeR. (2005). Modeling structures of intelligence. In WilhelmO.EngleR. W. (Eds.), Handbook of understanding and measuring intelligence (pp. 241-263). Thousand Oaks, CA: Sage.
68.
SliterK. A.ZickarM. J. (2014). An IRT examination of the psychometric functioning of negatively worded personality items. Educational and Psychological Measurement, 74, 214-226. doi:10.1177/0013164413504584
69.
SotoC. J.JohnO. P.GoslingS. D.PotterJ. (2008). The developmental psychometrics of Big Five self-reports: Acquiescence, factor structure, coherence, and differentiation from ages 10 to 20. Journal of Personality and Social Psychology, 94, 718-737. doi:10.1037/0022-3514.94.4.718
70.
SteinhauerH. W.AßmannC.ZinnS.GoßmannS.RässlerS. (2015). Sampling and weighting cohort samples in institutional contexts. AStA Wirtschafts-und Sozialstatistisches Archiv, 9, 131-157. doi:10.1007/s11943-015-0162-0
71.
TomásJ. M.OliverA. (1999). Rosenberg’s Self-Esteem Scale: Two factors or method effects. Structural Equation Modeling, 6, 84-98. doi:10.1080/10705519909540120
72.
TomásJ. M.OliverA.GalianaL.SanchoP.LilaM. (2013). Explaining method effects associated with negatively worded items in trait and state global and domain-specific self-esteem scales. Structural Equation Modeling, 20, 299-313. doi:10.1080/10705511.2013.769394
73.
van de SchootR.LugtigP.HoxJ. (2012). A checklist for testing measurement invariance. European Journal of Developmental Psychology, 9, 486-492. doi:10.1080/17405629.2012.686740
74.
von CollaniG.HerzbergP. Y. (2003a). Eine revidierte Fassung der deutschsprachigen Skala zum Selbstwertgefühl von Rosenberg [A revised version of the German adaptation of Rosenberg’s Self-Esteem Scale]. Zeitschrift für Differentielle und Diagnostische Psychologie, 24, 3-7. doi:10.1024//0170-1789.24.1.3
75.
von CollaniG.HerzbergP. Y. (2003b). Zur internen Struktur des globalen Selbstwertgefühls nach Rosenberg [On the internal structure of global self-esteem (Rosenberg)]. Zeitschrift für Differentielle und Diagnostische Psychologie, 24, 9-22. doi:10.1024//0170-1789.24.1.9
76.
WarmT. A. (1989). Weighted likelihood estimation of ability in item response theory. Psychometrika, 54, 427-450. doi:10.1007/BF02294627
77.
WeemsG. H.OnwuegbuzieA. J.CollinsK. M. (2006). The role of reading comprehension in responses to positively and negatively worded items on rating scales. Evaluation & Research in Education, 19, 3-20. doi:10.1080/09500790608668322
78.
WichertsJ. M.DolanC. V. (2010). Measurement invariance in confirmatory factor analysis: An illustration using IQ test performance of minorities. Educational Measurement: Issues and Practice, 29, 39-47. doi:10.1111/j.1745-3992.2010.00182.x
79.
WilhelmO. (2005). Measuring reasoning ability. In WilhelmO.EngleR. W. (Eds.), Handbook of understanding and measuring intelligence (pp. 373-392). Thousand Oaks, CA: Sage.
80.
WilliamsS. A.SwansonM. S. (2001). The effect of reading ability and response formats on patients’ abilities to respond to a patient satisfaction scale. Journal of Continuing Education in Nursing, 32, 60-67.
81.
WolfE. J.HarringtonK. M.ClarkS. L.MillerM. W. (2013). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety. Educational and Psychological Measurement, 73, 913-934. doi:10.1177/0013164413495237
82.
WuC. H. (2008). An examination of the wording effect in the Rosenberg Self-Esteem Scale among culturally Chinese people. Journal of Social Psychology, 148, 535-552. doi:10.3200/SOCP.148.5.535-552
83.
WuY.ZuoB.WenF.YanL. (2017). Rosenberg Self-Esteem Scale: Method effects, factorial structure and scale invariance across migrant child and urban child populations in China. Journal of Personality Assessment, 99, 83-93. doi:10.1080/00223891.2016.1217420
84.
YuanK.BentlerP. M. (2000). Three likelihood-based methods for mean and covariance structure analysis with nonnormal missing data. Sociological Methodology, 30, 167-202. doi:10.1111/0081-1750.00078
85.
ZuckermanM.LiC.HallJ. A. (2016). When men and women differ in self-esteem and when they don’t: A meta-analysis. Journal of Research in Personality, 64, 34-51. doi:10.1016/j.jrp.2016.07.007
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
