The essence of measurement invariance (MI) analysis is to test the assumption that observed scores on a scale accurately reflect respondents’ standings on a measured construct. Based on exploratory structural equation modeling, the current study examines gender-based MI in two Big Five measures of personality: the Mini-IPIP, and the Big Five Inventory (BFI) facet scales. We report results for MI based on both model fit indices and a practical significance index that quantifies the extent of noninvariance (i.e., dMACS). From the latter, we partition the observed group mean differences in scale scores into construct-irrelevant group differences versus construct-relevant group differences. In measures of Agreeableness, Conscientiousness, Extraversion, and Neuroticism across instruments, results supported metric invariance but not scalar invariance. That said, findings of statistical noninvariance were generally small in terms of practical effects, although some notable variability in the effects was evident. Overall, the current results provide evidence regarding gender-based MI of Big Five personality measures that are more detailed than that provided in past work. More generally, this study also provides useful guidance for future researchers investigating both the statistical and practical significance of measurement invariance.
AlmagorM.TellegenA.WallerN. G. (1995). The Big Seven model: A cross-cultural replication and further exploration of the basic dimensions of natural language trait descriptors. Journal of Personality and Social Psychology, 69, 300-307. doi:10.1037/0022-3514.69.2.300
2.
AndrichD.HagquistC. (2012). Real and artificial differential item functioning. Journal of Educational and Behavioral Statistics, 37, 387-416.
BakanD. (1966). The duality of human existence: An essay on psychology and religion. Chicago, IL: Rand McNally.
5.
BaldasaroR. E.ShanahanM. J.BauerD. J. (2013). Psychometric properties of the Mini-IPIP in a large, nationally representative sample of young adults. Journal of Personality Assessment, 95, 74-84. doi:10.1080/00223891.2012.700466
6.
BidermanM. D.McAbeeS. T.ChenZ. J.HendyN. T. (2018). Assessing the evaluative content of personality questionnaires using bifactor models. Journal of Personality Assessment, 100, 375-388. doi:10.1080/00223891.2017.1406362
7.
BoothT.HughesD. J. (2014). Exploratory structural equation modeling of personality data. Assessment, 21, 260-271. doi:10.1177/1073191114528029
8.
BorkenauP.OstendorfF. (1990). Comparing exploratory and confirmatory factor analysis: A study on the 5-factor model of personality. Personality and Individual Differences, 11, 515-524. doi:10.1016/0191-8869(90)90065-Y
9.
BrannickM. T. (1995). Critical comments on applying covariance structure modeling. Journal of Organizational Behavior, 16, 201-213. doi:10.1002/job.4030160303
10.
ByrneB. M.ShavelsonR. J.MuthénB. O. (1989). Testing for equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466. doi:10.1037/0033-2909.105.3.456
11.
ChangL.ConnellyB. S.GeezaA. A. (2012). Separating method factors and higher order traits of the Big Five: A meta-analytic multitrait-multimethod approach. Journal of Personality and Social Psychology, 102, 408-426. doi:10.1037/a0025559
12.
ChenF. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14, 464-504. doi:10.1080/10705510701301834
13.
CheungG. W.RensvoldR. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9, 233-255. doi:10.1207/S15328007SEM0902_5
14.
ChiorriC.MarshH. W.UbbialiA.DonatiD. (2016). Testing the factor structure and measurement invariance across gender of the Big Five Inventory through exploratory structural equation modeling. Journal of Personality Assessment, 98, 88-99. doi:10.1080/00223891.2015.1035381
15.
ChurchA. T.BurkeP. J. (1994). Exploratory and confirmatory tests of the Big Five and Tellegen’s three- and four-dimensional models. Journal of Personality and Social Psychology, 66, 93-114. doi:10.1037/0022-3514.66.1.93
16.
ColeD. A.CieslaJ. A.SteigerJ. H. (2007). The insidious effects of failing to include design-driven correlated residuals in latent-variable covariance structure analysis. Psychological Methods, 12, 381-398. doi:10.1037/1082-989X.12.4.381
17.
ConnellyB. S.OnesD. S.ChernyshenkoO. S. (2014). Introducing the special section on openness to experience: Review of openness taxonomies, measurement, and nomological net. Journal of Personality Assessment, 96, 1-16. doi:10.1016/j.paid.2011.09.015
18.
CooperA. J.SmillieL. D.CorrP. J. (2010). A confirmatory factor analysis of the Mini-IPIP five-factor model personality scale. Personality and Individual Differences, 48, 688-691. doi:10.1016/j.paid.2010.01.004
19.
CostaP. T.Jr.McCraeR. R. (1992). Revised NEO Personality Inventory (NEO-PI-R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources.
20.
CostaP. T.Jr.TerraccianoA.McCraeR. R. (2001). Gender differences in personality traits across cultures: Robust and surprising findings. Journal of Personality and Social Psychology, 81, 322-331. doi:10.1037/0022-3514.81.2.322
21.
DeYoungC. G.QuiltyL. C.PetersonJ. B. (2007). Between facets and domains: 10 Aspects of the Big Five. Journal of Personality and Social Psychology, 93, 880-896. doi:10.1037/0022-3514.93.5.880
22.
DonnellanM. B.OswaldF. L.BairdB. M.LucasR. E. (2006). The Mini-IPIP scales: Tiny-yet-effective measures of the Big Five factors of personality. Psychological Assessment, 18, 192-203. doi:10.1037/1040-3590.18.2.192
23.
EhrhartM. G.EhrhartK. M.AliA. A.RoeschS. C.PalmerL. (2012, April). A MIMIC model of acculturation and the International Personality Item Pool Five-Factor Model Measure. Poster presented at the 27th Annual Conference of the Society for Industrial and Organizational Psychology, San Diego, CA.
24.
EndersC. K.BandalosD. L. (2001). The relative performance of full information maximum likelihood estimation for missing data in structural equation models. Structural Equation Modeling, 8, 430-457. doi:10.1207/S15328007SEM0803_5
25.
FeingoldA. (1994). Gender differences in personality: A meta-analysis. Psychological Bulletin, 116, 429-456. doi:10.1037/0033-2909.116.3.429
26.
FurnhamA.GuenoleN.LevineS. Z.Chamorro-PremuzicT. (2013). The NEO Personality Inventory–Revised: Factor structure and gender invariance from exploratory structural equation modeling analyses in a high-stakes setting. Assessment, 20, 14-23. doi:10.1177/1073191112448213
27.
GignacG. E. (2009). Partial confirmatory factor analysis: Described and illustrated on the NEO PI-R. Journal of Personality Assessment, 91, 40-47. doi:10.1080/00223890802484126
28.
GoldbergL. R.JohnsonJ. A.EberH. W.HoganR.AshtonM. C.CloningerC. R.GoughH. G. (2006). The International Personality Item Pool and the future of public-domain personality measures. Journal of Research in Personality, 40, 84-96. doi:10.1016/j.jrp.2005.08.007
29.
GoldbergL. R.VelicerW. F. (2006). Principles of exploratory factor analysis. In StarkS. (Ed.), Differentiating normal and abnormal personality (2nd ed., pp. 209-237). New York, NY: Springer.
30.
HattieJ. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, 139-164. doi:10.1177/014662168500900204
31.
HopwoodC. J.DonnellanM. B. (2010). How should the internal structure of personality inventories be evaluated?Personality and Social Psychology Review, 14, 332-346. doi:10.1177/1088868310361240
32.
HoughL. M.FurnhamA. (2003). Use of personality variables in work settings. In BormanW. C.IlgenD. R.KlimoskiR. J. (Eds.), Handbook of psychology: Vol. 12. Industrial and organizational psychology (pp. 131-169). Hoboken, NJ: Wiley.
33.
HoughL. M.OnesD. S. (2001). The structure, measurement, validity, and use of personality variables in industrial, work, and organizational psychology. In AndersonN.OnesD. S.SinangilH. K.ViswesvaranC. (Eds.), Handbook of industrial and organizational psychology (Vol. 1, pp. 233-377). London, England: Sage.
34.
HoughL. M.OswaldF. L.PloyhartR. E. (2001). Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment, 9, 152-194. doi:10.1111/1468-2389.00171
35.
HuL.BentlerP. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55. doi:10.1080/10705519909540118
36.
JamshidianM.JalalS. (2010). Tests of homoscedasticity, normality, and missing completely at random for incomplete multivariate data. Psychometrika, 75, 649-674. doi:10.1007/s11336-010-9175-3
37.
JohnO. P.DonahueE. M.KentleR. L. (1991). The Big Five Inventory—Versions 4a and 54. Berkeley: University of California.
38.
KajoniusP. J.JohnsonJ. (2018). Sex differences in 30 facets of the five factor model of personality in the large public (N = 320,128). Personality and Individual Differences, 129, 126-130. doi:10.1016/j.paid.2018.03.026
39.
KellowayE. K. (1995). Structural equation modeling in perspective. Journal of Organizational Behavior, 16, 215-224. doi:10.1002/job.4030160304
40.
LaiM. H. C.KwokO. M.YoonM.HsiaoY. Y. (2017). Understanding the impact of partial factorial invariance on selection accuracy: An R script. Structural Equation Modeling, 24, 783-799. doi:10.1080/10705511.2017.1318703
41.
LanceC. E.ButtsM. M.MichelsL. C. (2006). The sources of four commonly reported cutoff criteria: What did they really say?Organizational Research Methods, 9, 202-220. doi:10.1177/1094428105284919
42.
LaverdièreO.MorinA. J. S.St-HilaireF. (2013). Factor structure and measurement invariance of a short measure of the Big Five personality traits. Personality and Individual Differences, 55, 739-743. doi:10.1016/j.paid.2013.06.008
43.
LimB. C.PloyhartR. E. (2006). Assessing the convergent and discriminant validity of Goldberg’s International Personality Item Pool: A multitrait-multimethod examination. Organizational Research Methods, 9, 29-54. doi:10.1177/1094428105283193
44.
LippaR. A. (2010). Gender differences in personality and interests: When, where, and why?Social and Personality Psychology Compass, 4, 1098-1110. doi:10.1111/j.1751-9004.2010.00320.x
45.
MarshH. W.LüdtkeO.MuthénB.AsparouhovT.MorinA. J. S.TrautweinU.NagengastB. (2010). A new look at the Big Five factor structures through exploratory structural equation modeling. Psychological Assessment, 22, 471-491. doi:10.1037/a0019227
46.
MarshH. W.MuthénB.AsparouhovT.LüdtkeO.RobitzschA.MorinA. J. S.TrautweinU. (2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to students’ evaluations of university teaching. Structural Equation Modeling, 16, 439-476. doi:10.1080/10705510903008220
47.
McCraeR. R.CostaP. T.Jr. (1985). Updating Norman’s “Adequate Taxonomy”: Intelligence and personality dimensions in natural language and in questionnaires. Journal of Personality and Social Psychology, 49, 710-721. doi:10.1037//0022-3514.49.3.710
48.
McCraeR. R.ZondermanA. B.CostaP. T.Jr.BondM. H.PaunonenS. V. (1996). Evaluating replicability of factors in the Revised NEO Personality Inventory: Confirmatory factor analysis versus Procrustes rotation. Journal of Personality and Social Psychology, 70, 552-566. doi:10.1037/0022-3514.70.3.552
49.
MeadeA. W.JohnsonE. C.BraddyP. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93, 568-592. doi:10.1037/0021-9010.93.3.568
MillsapR. E.KwokO. M. (2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 9, 93-115. doi:10.1037/1082-989X.9.1.93
52.
Mullins-SweattS. N.JamersonJ. E.SamuelD. B.OlsonD. R.WidigerT. A. (2006). Psychometric properties of an abbreviated instrument of the five-factor model. Assessment, 13, 119-137. doi:10.1177%2F1073191106286748
53.
MuthénB.AsparouhovT. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17, 313-335. doi:10.1037/a0026802
54.
MuthénL. K.MuthénB. O. (2017). Mplus user’s guide (8th ed.). Los Angeles, CA: Muthén & Muthén.
55.
NyeC. D.AllemandM.GoslingS. D.PotterJ.RobertsB. W. (2016). Personality trait differences between young and middle-aged adults: Measurement artifacts or actual trends?Journal of Personality, 84, 473-492. doi:10.1111/jopy.12173
56.
NyeC. D.BradburnJ.OlenickJ.BialkoC.DrasgowF. (2018). How big are my effects? Examining the magnitude of effect sizes in studies of measurement equivalence. Organizational Research Methods, 22, 678-709. doi:10.1177/1094428118761122
57.
NyeC. D.DrasgowF. (2011). Effect size indices for analyses of measurement equivalence: Understanding the practical importance of difference between groups. Journal of Applied Psychology, 96, 966-980. doi:10.1037/a0022955
58.
PloyhartR. E.OswaldF. L. (2004). Applications of mean and covariance structure analysis: Integrating correlational and experimental approaches. Organizational Research Methods, 7, 27-65. doi:10.1177/1094428103259554
59.
RajuN. S.LaffitteL. J.ByrneB. M. (2002). Measurement equivalence: A comparison of methods based on confirmatory factor analysis and item response theory. Journal of Applied Psychology, 87, 517-529. doi:10.1037/0021-9010.87.3.517
60.
RajuN. S.van der LindenW. J.FleerP. F. (1995). An IRT-based internal measure of test bias with applications for differential item functioning. Applied Psychological Measurement, 19, 353-368. doi:10.1177/014662169501900405
61.
SamuelD. B.SouthS. C.GriffinS. A. (2015). Factorial invariance of the five-factor model rating form across gender. Assessment, 22, 65-75. doi:10.1177%2F1073191114536772
62.
SchmittD. P.AllikJ.McCraeR. R.Benet-MartínezV. (2007). The geographical distribution of Big Five personality traits: Patterns and profiles of human self-descriptions across 56 nations. Journal of Cross-Cultural Psychology, 38, 173-212. doi:10.1177/0022022106297299
63.
SchmittD. P.RealoA.VoracekM.AllikJ. (2008). Why can’t a man be more like a woman? Sex differences in Big Five personality traits across 55 cultures. Journal of Personality and Social Psychology, 94, 168-182. doi:10.1037/0022-3514.94.1.168
64.
SchmittN.AliA. A. (2015). The practical importance of measurement invariance. In LanceC. E.VandenbergR. J. (Eds.), More statistical and methodological myths and urban legends (pp. 327-346). New York: NY: Routledge.
65.
SchmittN.GolubovichJ.LeongF. T. L. (2011). Measurement invariance and construct correlations, mean differences, and relationships with external correlates. Assessment, 18, 412-427. doi:10.1177/1073191110373223
66.
SchmittN.KuljaninG. (2008). Measurement invariance: Review of practice and implications. Human Resource Management Review, 18, 210-222. doi:10.1016/j.hrmr.2008.03.003
67.
ShchebetenkoS.KaluginA. Y.MishkevichA. M.SotoC. J.JohnO. P. (2019). Measurement invariance and sex and age differences of the Big Five Inventory-2: Evidence from the Russian versions. Assessment. Advance online publication. doi:10.1177/1073191119860901
68.
SochaA.CooperC. A.McCordD. M. (2010). Confirmatory factor analysis of the M5-50: An implementation of the International Personality Item Pool item set. Psychological Assessment, 22, 43-49. doi:10.1037/a0017371
69.
SotoC. J.JohnO. P. (2009). Ten facet scales for the Big Five Inventory: Convergence with NEO PI-R facets, self-peer agreement, and discriminant validity. Journal of Research in Personality, 43, 84-90. doi:10.1016/j.jrp.2008.10.002
70.
SotoC. J.JohnO. P. (2017). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power. Journal of Personality and Social Psychology, 113, 117-143. doi:10.1037/pspp0000096
71.
StarkS.ChernyshenkoO. S.DrasgowF. (2006). Detecting differential item functioning with CFA and IRT: Toward a unified strategy. Journal of Applied Psychology, 91, 1292-1306. doi:10.1037/0021-9010.91.1.25
72.
VandenbergR. J.LanceC. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 2, 4-70. doi:10.1177/109442810031002
73.
VassendO.SkrondalA. (1997). Validation of the NEO Personality Inventory and the five-factor model: Can findings from exploratory and confirmatory factor analysis be reconciled?European Journal of Personality, 11, 147-166. doi:10.1002/(SICI)1099-0984(199706)11:2%3C147::AID-PER278%3E3.0.CO;2-E
74.
WeisbergY. J.DeYoungC. G.HirshJ. B. (2011). Gender differences in personality across the ten aspects of the Big Five. Frontiers in Psychology, 2, 178. doi:10.3389/fpsyg.2011.00178
75.
WidamanK. F.LittleT. D.PreacherK. J.SawalaniG. M. (2011). On creating and using short forms of scales in secondary research. In TrzesniewskiK. H.DonnellanM. B.LucasR. E. (Eds.), Secondary data analysis: An introduction for psychologists (pp. 39-61). Washington, DC: American Psychological Association.
76.
ZimprichD.AllemandM.LachmanM. E. (2012). Factorial structure and age-related psychometrics of the MIDUS personality adjective items across the life span. Psychological Assessment, 24, 173-186. doi:10.1037/a0025265
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
