Factor mixture modeling (FMM) has been increasingly used to investigate unobserved population heterogeneity. This study examined the issue of covariate effects with FMM in the context of measurement invariance testing. Specifically, the impact of excluding and misspecifying covariate effects on measurement invariance testing and class enumeration was investigated via Monte Carlo simulations. Data were generated based on FMM models with (1) a zero covariate effect, (2) a covariate effect on the latent class variable, and (3) covariate effects on both the latent class variable and the factor. For each population model, different analysis models that excluded or misspecified covariate effects were fitted. Results highlighted the importance of including proper covariates in measurement invariance testing and evidenced the utility of a model comparison approach in searching for the correct specification of covariate effects and the level of measurement invariance. This approach was demonstrated using an empirical data set. Implications for methodological and applied research are discussed.
AkaikeH. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716-723. https://doi.org/10.1109/TAC.1974.1100705
2.
AllanN. P.MacPhersonL.YoungK. C.LejuezC. W.SchmidtN. B. (2014). Examining the latent structure of anxiety sensitivity in adolescents using factor mixture modeling. Psychological Assessment, 26(3), 741-751. https://doi.org/10.1037/a0036744
3.
AllanN. P.RainesA. M.CapronD. W.NorrA. M.ZvolenskyM. J.SchmidtN. B. (2014). Identification of anxiety sensitivity classes and clinical cut-scores in a sample of adult smokers: Results from a factor mixture model. Journal of Anxiety Disorders, 28(7), 696-703. https://doi.org/10.1016/j.janxdis.2014.07.006
4.
AsmundsonG. J. G.TaylorS.CarletonR. N.WeeksJ. W.HadjstavropoulosH. D. (2012). Should health anxiety be carved at the joint? A look at the health anxiety construct using factor mixture modeling in a non-clinical sample. Journal of Anxiety Disorders, 26(1), 246-251. https://doi.org/10.1016/j.janxdis.2011.11.009
5.
AsparouhovT.MuthénB. (2014). Auxiliary variables in mixture modeling: Three-step approaches using Mplus. Structural Equation Modeling, 21(3), 329-341. https://doi.org/10.1080/10705511.2014.915181
6.
BabusaB.CzeglédiE.TúryF.MayvilleS. B.UrbánR. (2015). Differentiating the levels of risk for muscle dysmorphia among Hungarian male weightlifters: A factor mixture modeling approach. Body Image, 12, 14-21. https://doi.org/10.1016/j.bodyim.2014.09.001
7.
BauerD. J. (2007). Observations on the use of growth mixture models in psychological research. Multivariate Behavioral Research, 42(4), 757-786. https://doi.org/10.1080/00273170701710338
8.
BauerD. J.CurranP. J. (2004). The integration of continuous and discrete latent variable models: Potential problems and promising opportunities. Psychological Methods, 9(1), 3-29. https://doi.org/10.1037/1082-989X.9.1.3
9.
BernsteinA.StickleT. R.SchmidtN. B. (2013). Factor mixture model of anxiety sensitivity and anxiety psychopathology vulnerability. Journal of Affective Disorders, 149(1-3), 406-417. https://doi.org/10.1016/j.jad.2012.11.024
10.
BozdoganH. (1987). Model selection and Akaike’s information criterion (AIC): The general theory and its analytical extensions. Psychometrika, 52(3), 345-370. https://doi.org/10.1007/BF02294361
CassadyJ. C.FinchW. H. (2015). Using factor mixture modeling to identify dimensions of cognitive test anxiety. Learning and Individual Differences, 41, 14-20. https://doi.org/10.1016/j.lindif.2015.06.002
13.
ChoS.CohenA. S. (2010). A multilevel mixture IRT model with an application to DIF. Journal of Educational and Behavioral Statistics, 35(3), 336-370. https://doi.org/10.3102/1076998609353111
14.
ClarkS. L.MuthénB. O.KaprioJ.D’OnofrioB. M.VikenR.RoseR. J. (2013). Models and strategies for factor mixture analysis: An example concerning the structural underlying psychological disorders. Structural Equation Modeling, 20(4), 681-703. https://doi.org/10.1080/10705511.2013.824786
15.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Academic Press.
16.
DavidovE.DülmerH.SchlüterE.SchmidtP.MeulemanB. (2012). Using a multilevel structural equation modeling approach to explain cross-cultural measurement noninvariance. Journal of Cross-Cultural Psychology, 43(4), 558-575. https://doi.org/10.1177/0022022112438397
17.
De AyalaR. J.KimS.-H.StapletonL. M.DaytonC. M. (2002). Differential item functioning: A mixture distribution conceptualization. International Journal of Testing, 2(3-4), 243-276. https://doi.org/10.1080/15305058.2002.9669495
18.
DiasJ. G. (2004). Finite mixture models: Review, applications, and computer-intensive methods. Ridderprint.
19.
DimitrovD. M.Al-SaudF. A. A.-M.AlsadaawiA. S. (2015). Investigating population heterogeneity and interaction effects of covariates: The case of a large-scale assessment for teacher licensure in Saudi Arabia. Journal of Psychoeducational Assessment, 33(7), 674-686. https://doi.org/10.1177/0734282914562121
20.
ElhaiJ. D.NaifehJ. A.ForbesD.RactliffeK. C.TamburrinoM. (2011). Heterogeneity in clinical presentations of posttraumatic stress disorder among medical patients: Testing factor structure variation using factor mixture modeling. Journal of Traumatic Stress, 24(4), 435-443. https://doi.org/10.1002/jts.20653
21.
FrostR.HylandP.McCarthyA.HalpinR.ShevlinM.MurphyJ. (2019). The complexity of trauma exposure and response: Profiling PTSD and CPTSD among a refugee sample. Psychological Trauma: Theory, Research, Practice, and Policy, 11(2), 165-175. https://doi.org/10.1037/tra0000408
22.
GrimmK. J.MazzaG. L.DavoudzadehP. (2016). Model selection in finite mixture models: A k-fold cross-validation approach. Structural Equation Modeling, 24(2), 246-256. https://doi.org/10.1080/10705511.2016.1250638
23.
HensonJ. M.ReiseS. P.KimK. H. (2007). Detecting mixtures from structural model differences using latent variable mixture modeling: A comparison of relative model fit statistics. Structural Equation Modeling, 14(2), 202-226. https://doi.org/10.1080/10705510709336744
24.
HuJ.LeiteW. L.GaoM. (2017). An evaluation of the use of covariates to assist in class enumeration in linear growth mixture modeling. Behavior Research Methods, 49(3), 1179-1190. https://doi.org/10.3758/s13428-016-0778-1
25.
JackmanM. G.-A. (2012). A Monte Carlo investigation of the performance of factor mixture modeling in the detection of differential item functioning (Publication No. 3480453) [Doctoral dissertation]. ProQuest Dissertation & Theses A&I.
26.
JacobucciR.GrimmK. J.McArdleJ. J. (2017). A comparison of methods for uncovering sample heterogeneity: Structural equation model trees and finite mixture models. Structural Equation Modeling, 24(2), 270-282. https://doi.org/10.1080/10705511.2016.1250637
27.
JakS.OortF. J.DolanC. V. (2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling, 20(2), 265-282. https://doi.org/10.1080/10705511.2013.769392
28.
KellerA. C.IgicI.MeierL. L.SemmerN. K.SchaubroeckJ. M.BrunnerB.ElferingA. (2016). Testing job typologies and identifying at-risk subpopulations using factor mixture models. Journal of Occupational Health Psychology, 22(4), 503-517. https://doi.org/10.1037/ocp0000038
29.
KimE. S.CaoC.WangY.NguyenD. (2017). Measurement invariance testing with many groups: A comparison of five approaches. Structural Equation Modeling, 24(4), 524-544. https://doi.org/10.1080/10705511.2017.1304822
KimE. S.WangY. (2018). Investigating sources of heterogeneity with 3-step multilevel factor mixture modeling: Beyond testing measurement invariance in cross-national studies. Structural Equation Modeling, 26(2), 165-181. https://doi.org/10.1080/10705511.2018.1521284
32.
KimE. S.YoonM. (2011). Testing measurement invariance: A comparison of multiple-group categorical CFA and IRT. Structural Equation Modeling, 18(2), 212-228. https://doi.org/10.1080/10705511.2011.557337
33.
KimM.VermuntJ.BakkZ.JakiT.Van HornM. L. (2016). Modeling predictors of latent classes in regression mixture models. Structural Equation Modeling, 23(4), 601-614. https://doi.org/10.1080/10705511.2016.1158655
34.
LeeH.BeretvasS. N. (2014). Evaluation of two types of differential item functioning in factor mixture models with binary outcomes. Educational and Psychological Measurement, 74(5), 831-858. https://doi.org/10.1177/0013164414526881
35.
LoY.MendellN. R.RubinD. B. (2001). Testing the number of components in a normal mixture. Biometrika, 88(3), 767-778. https://doi.org/10.1093/biomet/88.3.767
36.
LubkeG.MuthénB. O. (2007). Performance of factor mixture models as a function of model size, covariate effects, and class-specific parameters. Structural Equation Modeling, 14(1), 26-47. https://doi.org/10.1080/10705510709336735
37.
LubkeG. H.MuthénB. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10(1), 21-39. https://doi.org/10.1037/1082-989X.10.1.21
38.
LubkeG.NealeM. C. (2006). Distinguishing between latent classes and continuous factors: Resolution by maximum likelihood?Multivariate Behavioral Research, 41(4), 499-532. https://doi.org/10.1207/s15327906mbr4104_4
39.
LubkeG.NealeM. C. (2008). Distinguishing between latent classes and continuous factors with categorical outcomes: Class invariance of parameters of factor mixture models. Multivariate Behavioral Research, 43(4), 592-620. https://doi.org/10.1080/00273170802490673
40.
LukočienėO.VarrialeR.VermuntJ. K. (2010). The simultaneous decision(s) about the number of lower- and higher-level classes in multilevel latent class analysis. Sociological Methodology, 40(1), 247-283. https://doi.org/10.1111/j.1467-9531.2010.01231.x
41.
LukočienėO.VermuntJ. K. (2010). Determining the number of components in mixture models for hierarchical data. In FinkA.BertholdL.SeidelW.UltschA. (Eds.), Advances in data analysis, data handling and business intelligence (pp. 241-249). Springer. https://doi.org/10.1007/978-3-642-01044-6_22
42.
Maij-de MeijA. M.KeldermanH.Van der FlierH. (2010). Improvement in detection of differential item functioning using a mixture item response theory model. Multivariate Behavioral Research, 45(6), 975-999. https://doi.org/10.1080/00273171.2010.533047
MuthénB. O. (2004). Latent variable analysis: Growth mixture modeling and related techniques. In KaplanD. (Ed.), Handbook of quantitative methodology for the social sciences (pp. 345-368). Sage.
45.
MuthénL. K.MuthénB. O. (1998-2014). Mplus user’s guide (7th ed.).
46.
NylundK. L.AsparouhovT.MuthénB. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling, 14(4), 535-569. https://doi.org/10.1080/10705510701575396
47.
Nylund-GibsonK.MasynK. E. (2016). Covariates and mixture modeling: Results of a simulation study exploring the impact of misspecified effects on class enumeration. Structural Equation Modeling, 23(6), 782-797. https://doi.org/10.1080/10705511.2016.1221313
48.
PeughJ.FanX. (2014). Enumeration index performance in generalized growth mixture models: A Monte Carlo test of Muthén’s (2003) hypothesis. Structural Equation Modeling, 22(1), 115-131. https://doi.org/10.1080/10705511.2016.1221313
ScloveS. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52(3), 333-343. https://doi.org/10.1007/BF02294360
51.
StegmannG.GrimmK. J. (2018). A new perspective on the effects of covariates in mixture models. Structural Equation Modeling, 25(2), 167-178. https://doi.org/10.1080/10705511.2017.1318070
52.
SubramaniamM.AbdinE.VaingankarJ. A.VermaS.ChongS. A. (2014). Latent structure of psychosis in the general population: Results from the Singapore Mental Health Study. Psychological Medicine, 44(1), 51-60. https://doi.org/10.1017/S0033291713000688
53.
TayL.NewmanD. A.VermuntJ. K. (2011). Using mixed-measurement item response theory with covariates (MM-IRT-C) to ascertain observed and unobserved measurement equivalence. Organizational Research Methods, 14(1), 147-176. https://doi.org/10.1177/1094428110366037
54.
TeinJ.CoxeS.ChamH. (2013). Statistical power to detect the correct number of classes in latent profile analysis. Structural Equation Modeling, 20(4), 640-657. https://doi.org/10.1080/10705511.2013.824781
55.
TofighiD.EndersC. (2008). Identifying the correct number of classes in a growth mixture model. In HancockG. (Ed.), Mixture models in latent variable research (pp. 317-341). Information Age.
56.
VermuntJ. K. (2010). Latent class modeling with covariates: Two improved three-step approaches. Political Analysis, 18(4), 450-469. https://doi.org/10.1093/pan/mpq025
57.
WangY.KimE.JooS.-H.ChunS.AlamriA.LeeP.StarkS. (2020) reconsidering multilevel latent class models: Can level-2 latent classes affect item response probabilities?Journal of Experimental Education. Advance online publication. https://doi.org/10.1080/00220973.2020.1737913
58.
YangC. C. (2006). Evaluating latent class analysis models in qualitative phenotype identification. Computational Statistics & Data Analysis, 50(4), 1090-1104. https://doi.org/10.1016/j.csda.2004.11.004
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
