We used exploratory structural equation modeling to examine gender-based measurement invariance (MI) in the HEXACO-100 across three samples that varied in terms of age (undergraduate students in Study 1, working adults in Studies 2 and 3) and testing context (research context in Studies 1 and 2, high-stakes selection context in Study 3). Across three studies, we consistently found support for configural and metric invariance but not scalar invariance. However, the effect size measures of non-invariance were generally small. That said, in the Emotionality scale, for the same latent score, females scored higher than males due to measurement non-invariance (between 0.26 and 0.48 standard deviation units). Thus, the observed mean gender differences overestimated the true mean gender differences. The current study provides detailed evidence regarding gender-based MI of HEXACO personality scales. More generally, it provides insight regarding the effect that measurement artifacts can have on understanding psychological gender differences at the latent level.
AnglimJ.MorseG.de VriesR. E.MacCannC.MartyA. (2017). Comparing job applicants to non-applicants using an item-level bifactor model on the HEXACO personality inventory. European Journal of Personality, 31(6), 669–684. https://doi.org/10.1002/per.2120
2.
AshtonM. C.LeeK.PeruginiM.SzarotaP.de VriesR. E.Di BlasL.BoiesK.De RaadB. (2004). A six-factor structure of personality-descriptive adjectives: Solutions from psycholexical studies in seven languages. Journal of Personality and Social Psychology, 86(2), 356–366. https://doi.org/10.1037/0022-3514.86.2.356
BirkelandS. A.MansonT. M.KisamoreJ. L.BrannickM. T.SmithM. A. (2006). A meta-analytic investigation of job applicant faking on personality measures. International Journal of Selection and Assessment, 14(4), 317–335. https://doi.org/10.1111/j.1468-2389.2006.00354.x
6.
BrannickM. T. (1995). Critical comment on applying covariance structure modeling. Journal of Organizational Behavior, 16(3), 201–213. https://doi.org/10.1002/job.4030160303
7.
ByrneB. M.ShavelsonR. J.MuthénB. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105(3), 456–466. https://doi.org/10.1037/0033-2909.105.3.456
8.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling: A Multidisciplinary Journal, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5
9.
ChiorriC.MarshH. W.UbbialiA.DonatiD. (2016). Testing the factor structure and measurement invariance across gender of the Big Five inventory through exploratory structural equation modeling. Journal of Personality Assessment, 98(1), 88–99. https://doi.org/10.1080/00223891.2015.1035381
10.
CostaP. T., JrTerraccianoA.McCraeR. R. (2001). Gender differences in personality traits across cultures: Robust and surprising findings. Journal of Personality and Social Psychology, 81(2), 322–331. https://doi.org/10.1037/0022-3514.81.2.322
11.
DeShonR. P. (2004). Measures are not invariant across groups without error variance homogeneity. Psychological Science, 46(1), 137–149.
EaglyA. H.WoodW. (1991). Explaining sex differences in social behavior: A meta-analytic perspective. Personality and Social Psychology Bulletin, 17(3), 306–315. https://doi.org/10.1177/0146167291173011
14.
EllingsonJ. E.SmithD. B.SackettP. R. (2001). Investigating the influence of social desirability on personality factor structure. Journal of Applied Psychology, 86(1), 122–133. https://doi.org/10.1037/0021-9010.86.1.122
FurnhamA.GuenoleN.LevineS. Z.Chamorro-PremuzicT. (2013). The NEO Personality Inventory-Revised: Factor structure and gender invariance from exploratory structural equation modeling analyses in a high-stakes setting. Assessment, 20(1), 14–23. https://doi.org/10.1177/1073191112448213
17.
GarcíaL. F.AlujaA.RossierJ.OstendorfF.GlicksohnJ.OumarB.BellajT.RuchW.WangW.KöviZ.ŚcigałaD.ČekrlijaĐ.StiversA. W.Di BlasL.ValdiviaM.Ben JemaaS.AtitsogbeK. A.HansenneM. (2021). Exploring the stability of HEXACO-60 structure and the association of gender, age, and social position with personality traits across 18 countries. Journal of Personality, 90(2), 256–276. https://doi.org/10.1111/jopy.12664
18.
GoldbergL. R. (1990). An alternative “description of personality”: The Big-Five factor structure. Journal of Personality and Social Psychology, 59(6), 1216–1229. https://doi.org/10.1037/0022-3514.59.6.1216
19.
GunnH. J.GrimmK. J.EdwardsM. C. (2020). Evaluation of six effect size measures of measurement non-invariance for continuous outcomes. Structural Equation Modeling: A Multidisciplinary Journal, 27(4), 503–514. https://doi.org/10.1080/10705511.2019.1689507
20.
HendriksM. C.CroonM. A.VingerhoetsA. J. (2008). Social reactions to adult crying: The help-soliciting function of tears. The Journal of Social Psychology, 148(1), 22–42. https://doi.org/10.3200/SOCP.148.1.22-42
21.
IonA.IliescuD.AldhafriS.RanaN.RatanadilokK.WidyantiA.NedelceaC. (2017). A cross-national analysis of personality structure through the lens of the HEXACO model. Journal of Personality Assessment, 99(1), 25–34. https://doi.org/10.1080/00223891.2016.1187155
22.
KellowayE. K. (1995). Structural equation modelling in perspective. Journal of Organizational Behavior, 16(3), 215–224. https://doi.org/10.1002/job.4030160304
23.
LaiM. H. C.ZhangY. (2022). Classification accuracy of multidimensional tests: Quantifying the impact of noninvariance. Structural Equation Modeling: A Multidisciplinary Journal, 29(4), 620–629. https://doi.org/10.1080/10705511.2021.1977936
LeeK.AshtonM. C. (2020). Sex differences in HEXACO personality characteristics across countries and ethnicities. Journal of Personality, 88(6), 1075–1090. https://doi.org/10.1111/jopy.12551
26.
LubkeG. H.DolanC. V.KeldermanH.MellenberghG. J. (2003). Weak measurement invariance with respect to unmeasured variables: An implication of strict factorial invariance. British Journal of Mathematical and Statistical Psychology, 56(2), 231–248. https://doi.org/10.1348/000711003770480020
27.
MarshH. W.LüdtkeO.MuthénB.AsparouhovT.MorinA. J. S.TrautweinU.NagengastB. (2010). A new look at the Big Five factor structures through exploratory structural equation modeling. Psychological Assessment, 22(3), 471–491. https://doi.org/10.1037/a0019227
28.
McCraeR. R. (2015). A more nuanced view of reliability: Specificity in the trait hierarchy. Personality and Social Psychology Review, 19(2), 97–112. https://doi.org/10.1177/1088868314541857
29.
MeadeA. W.JohnsonE. C.BraddyP. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568–592. https://doi.org/10.1037/0021-9010.93.3.568
30.
MillsapR. E.KwokO.-M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9(1), 93–115. https://doi.org/10.1037/1082-989X.9.1.93
31.
MonseurC.BereznerA. (2007). The computation of equating errors in international surveys in education. Journal of Applied Measurement, 8(3), 323–335.
32.
MoshagenM.HilbigB. E.ZettlerI. (2014). Factor structure, psychometric properties, and measurement invariance of the German-language version of the 60-item HEXACO Personality Inventory, Diagnostica, 60(2), 86–97. https://doi.org/10.1026/0012-1924/a000112
33.
MoshagenM.ThielmannI.HilbigB. E.ZettlerI. (2019). Meta-analytic investigations of the HEXACO Personality Inventory (-Revised): Reliability generalization, self-observer agreement, intercorrelations, and relations to demographic variables. Zeitschrift für Psychologie, 227(3), 186–194. https://doi.org/10.1027/2151-2604/a000377
34.
MuthénB.AsparouhovT. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17(3), 313–335. https://doi.org/10.1037/a0026802
35.
MuthénL. K.MuthénB. O. (2017). Mplus user’s guide (8th ed.).
36.
NyeC. D.BradburnJ.OlenickJ.BialkoC.DrasgowF. (2019). How big are my effects? Examining the magnitude of the effect sizes in studies of measurement equivalence. Organizational Research Methods, 22(3), 678–709. https://doi.org/10.1177/1094428118761122
37.
NyeC. D.DrasgowF. (2011). Effect size indices for analyses of measurement equivalence: Understanding the practical importance of differences between groups. Journal of Applied Psychology, 96, 966–980. https://doi.org/10.1037/a0022955
38.
OckJ.McAbeeS. T.MulfingerE.OswaldF. L. (2020). The practical effects of measurement invariance: Gender invariance in two Big Five personality measures. Assessment, 27(4), 657–674. https://doi.org/10.1177/1073191119885018
39.
PloyhartR. E.OswaldF. L. (2004). Applications of mean and covariance structure analysis: Integrating correlational and experimental approaches. Organizational Research Methods, 7(1), 27–65. https://doi.org/10.1177/1094428103259554
40.
RobertsB. W.WaltonK. E.ViechtbauerW. (2006). Patterns of mean-level change in personality traits across the life course: A meta-analysis of longitudinal studies. Psychological Bulletin, 132(1), 1–25. https://doi.org/10.1037/0033-2909.132.1.1
41.
RobitzschA.LüdtkeO. (2019). Linking errors in international large-scale assessments: Calculation of standard errors for trend estimation. Assessment in Education: Principles, Policy & Practice, 26(4), 444–465. https://doi.org/10.1080/0969594X.2018.1433633
42.
RobitzschA.LüdtkeO. (2022). Why measurement invariance is not necessary for valid group comparisons. PsyArXiv. https://doi.org/10.31234/osf.io/cjyqp
43.
SamuelD. B.SouthS. C.GriffinS. A. (2015). Factorial invariance of the five-factor model rating form across gender. Assessment, 22(1), 65–75. https://doi.org/10.1177/1073191114536772
44.
SchmittD. P.RealoA.VoracekM.AllikJ. (2008). Why can’t a man be more like a woman? Sex differences in Big Five personality traits across 55 cultures. Journal of Personality and Social Psychology, 94(1), 168–182. https://doi.org/10.1037/0022-3514.94.1.168
45.
SchmittN.AliA. A. (2015). The practical importance of measurement invariance. In LanceC. E.VandenbergR. J. (Eds.), More statistical and methodological myths and urban legends (pp. 327–346). Routledge/Taylor & Francis Group.
46.
SchmittN.GolubovichJ.LeongF. T. L. (2011). Impact of measurement invariance on construct correlations, mean differences, and relations with external correlates: An illustrative example using Big Five and RIASEC measures. Assessment, 18(4), 412–427. https://doi.org/10.1177/1073191110373223
47.
SomarajuA. V.NyeC. D.OlenickJ. (2022). A review of measurement equivalence in organizational research: What’s old, what’s new, and what’s next?Organizational Research Methods, 25(4), 741–785. https://doi.org/10.1177/10944281211056524
48.
StarkS.ChernyshenkoO. S.DrasgowF. (2006). Detecting differential item functioning with confirmatory factor analysis and item response theory: Toward a unified strategy. Journal of Applied Psychology, 91(6), 1292–1306. https://doi.org/10.1037/0021-9010.91.6.1292
49.
ThielmannI.AkramiN.BabarovićT.BellochA.BerghR.ChirumboloA.ČolovićP.de VriesR. E.DostálD.EgorovaM.GnisciA.HeydaschT.HilbigB. E.HsuK.-Y.IzdebskiP.LeoneL.MarcusB.MededovićJ.NagyJ.. . .LeeK. (2020). The HEXACO-100 across 16 languages: A large-scale test of measurement invariance. Journal of Personality Assessment, 102(5), 714–726. https://doi.org/10.1080/00223891.2019.1614011
50.
VandenbergR. J.LanceC. E. (2000). A review and synthesis on the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3(1), 4–69. https://doi.org/10.1177/109442810031002
51.
WuH.BrowneM. W. (2015). Quantifying adventitious error in a covariance structure as a random effect. Psychometrika, 80(3), 571–600. https://doi.org/10.1007/s11336-015-9451-3
