Although collecting data from multiple informants is highly recommended, methods to model the congruence and incongruence between informants are limited. Bauer and colleagues suggested the trifactor model that decomposes the variances into common factor, informant perspective factors, and item-specific factors. This study extends their work to the trifactor mixture model that combines the trifactor model and the mixture model. This combined approach allows researchers to investigate the common and unique perspectives of multiple informants on targets using latent factors and simultaneously take into account potential heterogeneity of targets using latent classes. We demonstrate this model using student self-rated and teacher-rated academic behaviors (N = 24,094). Model specification and testing procedures are explicated in detail. Methodological and practical issues in conducting the trifactor mixture analysis are discussed.
AchenbachT. M. (2011). Commentary. Definitely more than measurement error: But how should we understand and deal with informant discrepancies?Journal of Clinical Child & Adolescent Psychology, 40(1), 80-86. https://doi.org/10.1080/15374416.2011.533416
2.
AchenbachT. M.KrukowskiR. A.DumenciL.IvanovaM. Y. (2005). Assessment of adult psychopathology: Meta-analyses and implications of cross-informant correlations. Psychological Bulletin, 131(3), 361-382. https://doi.org/10.1037/0033-2909.131.3.361
3.
AchenbachT. M.McConaughyS. H.HowellC. T. (1987). Child/adolescent behavioral and emotional problems: Implications of cross-informant correlations for situational specificity. Psychological Bulletin, 101(2), 213-232. https://doi.org/10.1037/0033-2909.101.2.213
4.
AkaikeH. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716-723. https://doi.org/10.1109/TAC.1974.1100705
5.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (2014). Standards for educational and psychological testing. Joint Committee on Standards for Educational and Psychological Testing.
6.
AsparouhovT.MuthénB. (2014). Auxiliary variables in mixture modeling: Three-step approaches using Mplus. Structural Equation Modeling: A Multidisciplinary Journal, 21(3), 329-341. https://doi.org/10.1080/10705511.2014.915181
7.
BauerD. J.HowardA. L.BaldasaroR. E.CurranP. J.HussongA. M.ChassinL.ZuckerR. A. (2013). A trifactor model for integrating ratings across multiple informants. Psychological Methods, 18(4), 475-493. https://doi.org/10.1037/a0032475
8.
BeauducelA.HerzbergP. Y. (2006). On the performance of maximum likelihood versus means and variance adjusted weighted least squares estimation in CFA. Structural Equation Modeling: A Multidisciplinary Journal, 13(2), 186-203. https://doi.org/10.1207/s15328007sem1302_2
9.
ClarkS. L.MuthénB. O.KaprioJ.D’OnofrioB. M.VikenR.RoseR. J. (2013). Models and strategies for factor mixture analysis: An example concerning the structural underlying psychological disorders. Structural Equation Modeling: A Multidisciplinary Journal, 20(4), 681-703. https://doi.org/10.1080/10705511.2013.824786
10.
CollinsL. M.LanzaS. T. (2010). Latent class and latent transition analysis: With applications in the social, behavioral, and health sciences. Wiley.
11.
De BoeckP.WilsonM.ActonG. S. (2005). A conceptual and psychometric framework for distinguishing categories and dimensions. Psychological Review, 112(1), 129-158. https://doi.org/10.1037/0033-295X.112.1.129
12.
De Los ReyesA. (2011). Introduction to the special section. More than measurement error: Discovering meaning behind informant discrepancies in clinical assessments of children and adolescents. Journal of Clinical Child & Adolescent Psychology, 40(1), 1-9. https://doi.org/10.1080/15374416.2011.533405
13.
DiStefanoC.MorganG. B. (2014). A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Structural Equation Modeling: A Multidisciplinary Journal, 21(3), 425-438. https://doi.org/10.1080/10705511.2014.915373
14.
EndersC. K.TofighiD. (2008). The impact of misspecifying class-specific residual variances in growth mixture models. Structural Equation Modeling: A Multidisciplinary Journal, 15(1), 75-95. https://doi.org/10.1080/10705510701758281
15.
FlakeJ. K.PetwayK. T.II., (2019). Methodologies for investigating and interpreting student–teacher rating incongruence in noncognitive assessment. Educational Measurement: Issues and Practice, 38(1), 63–77. https://doi.org/10.1111/emip.12201
16.
IaccarinoS.von der EmbseN. P.KilgusS. P. (2019). Interpretation and use of the Social, Academic, and Emotional Behavior Risk Screener: A latent transition approach. Journal of Psychoeducational Assessment, 37(4), 486-503. https://doi.org/10.1177/0734282918766650
17.
JooS.-H.KimE. S. (2019). Impact of the strict invariance violations when testing a group mean difference and measurement invariance using MIMIC. Behavior Research Methods, 51(6), 2688-2699. https://doi.org/10.3758/s13428-018-1124-6
18.
JöreskogK. G. (1969). A general approach to confirmatory maximum likelihood factor analysis. Psychometrika, 34, 183-202. https://doi.org/10.1007/BF02289343
19.
KilgusS. P.SimsW.von der EmbseN. P.Riley-TillmanT. C. (2015). Confirmation of models for interpretation and use of the Social and Academic Behavior Risk Screener (SABRS). School Psychology Quarterly, 30(3), 335-352. https://doi.org/10.1037/spq0000087
20.
KilgusS. P.von der EmbseN. P. (2014). The Social, Academic, and Emotional Behavior Risk Screener—Teacher Rating Scale [Measurement instrument]. FastBridge Learning.
21.
KimE. S.CaoC.WangY.NguyenD. (2017). Measurement invariance testing with many groups: A comparison of five approaches. Structural Equation Modeling: A Multidisciplinary Journal, 24(4), 524-544. https://doi.org/10.1080/10705511.2017.1304822
22.
KimE. S.JooS.-H.LeeP.WangY.StarkS. (2016). Measurement invariance testing across between-level latent classes using multilevel factor mixture modeling. Structural Equation Modeling: A Multidisciplinary Journal, 23(6), 870-887. https://doi.org/10.1080/10705511.2016.1196108
23.
KimE. S.WangY.KieferS. M. (2018). Cross-level group measurement invariance when groups are at different levels of multilevel data. Educational and Psychological Measurement, 78(6), 973-997. https://doi.org/10.1177/0013164417739062
24.
KimE. S.WangY. (2019). Investigating sources of heterogeneity with 3-step multilevel factor mixture modeling: Beyond testing measurement invariance in cross-national studies. Structural Equation Modeling: A Multidisciplinary Joural, 26(2), 165-181. https://doi.org/10.1080/10705511.2018.1521284
25.
KraemerH. C.MeaselleJ. R.AblowJ. C.EssexM. J.BoyceW. T.KupferD. J. (2003). A new approach to integrating data from multiple informants in psychiatric assessment and research: Mixing and matching contexts and perspectives. American Journal of Psychiatry, 160(9), 1566-1577. doi: 10.1176/appi.ajp.160.9.1566
26.
LairdR. D.De Los ReyesA. (2013). Testing informant discrepancies as predictors of early adolescent psychopathology: Why difference scores cannot tell you what you want to know and how polynomial regressionmay. Journal of Abnormal Child Psychology, 41(1), 1-14. https://doi.org/10.1007/s10802-012-9659-y
27.
LeiP.-W. (2009). Evaluating estimation methods for ordinal data in structural equation modeling. Quality and Quantity, 43, Article 495. https://doi.org/10.1007/s11135-007-9133-z
28.
LubkeG. H.MuthénB. (2005). Investigating population heterogeneity with factor mixture models. Psychological Methods, 10(1), 21-39. https://doi.org/10.1037/1082-989X.10.1.21
29.
LubkeG.MuthénB. O. (2007). Performance of factor mixture models as a function of model size, covariate effects, and class-specific parameters. Structural Equation Modeling: A Multidisciplinary Journal, 14(1), 26-47. https://doi.org/10.1080/10705510709336735
30.
LubkeG.NealeM. C. (2006). Distinguishing between latent classes and continuous factors: Resolution by maximum likelihood?Multivariate Behavioral Research, 41(4), 499-532. https://doi.org/10.1207/s15327906mbr4104_4
31.
LubkeG.NealeM. C. (2008). Distinguishing between latent classes and continuous factors with categorical outcomes: Class invariance of parameters of factor mixture models. Multivariate Behavioral Research, 43(4), 592-620. https://doi.org/10.1080/00273170802490673
32.
LukočienėO.VermuntJ. K. (2010). Determining the number of components in mixture models for hierarchical data. In FinkA.BertholdL.SeidelW.UltschA. (Eds.), Advances in data analysis, data handling and business intelligence (pp. 241-249). Springer. https://doi.org/10.1007/978-3-642-01044-6_22
33.
MacCallumR. C.ZhangS.PreacherK. J.RuckerD. D. (2002). On the practice of dichotomization of quantitative variables. Psychological Methods, 7(1), 19-40. https://doi.org/10.1037/1082-989X.7.1.19
34.
McDonaldR. P. (1967). Non-linear factor analysis. Psychometric Monographs, 15, 167.
McNeishD.HarringJ. (2020). Covariance pattern mixture models: Eliminating random effects to improve convergence and performance. Behavior Research Methods, 52, 947-979. https://doi.org/10.3758/s13428-019-01292-4
37.
MorinA. J. S.MaÿanoC.NagengastB.MarshH. W.MorizotJ.JanoszM. (2011). General growth mixture analysis of adolescents’ developmental trajectories of anxiety: The impact of untested invariance assumptions on substantive interpretations. Structural Equation Modeling, 18(4), 613-648. doi: 10.1080/10705511.2011.607714
NylundK. L.AsparouhovT.MuthénB. O. (2007). Deciding on the number of classes in latent class analysis and growth mixture modeling: A Monte Carlo simulation study. Structural Equation Modeling: A Multidisciplinary Journal, 14(4), 535-569. https://doi.org/10.1080/10705510701575396
41.
Nylund-GibsonK.MasynK. E. (2016). Covariates and mixture modeling: Results of a simulation study exploring the impact of misspecified effects on class enumeration. Structural Equation Modeling: A Multidisciplinary Journal, 23(6), 782-797. https://doi.org/10.1080/10705511.2016.1221313
42.
RhemtullaM.Brosseau-LiardP. E.SavaleiV. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354-373. https://doi.org/10.1037/a0029315
43.
RoskamI. (2018). The clinical significance of informant agreement in externalizing behavior from age 3 to 14. Child Psychiatry & Human Development, 49, 563-571. https://doi.org/10.1007/s10578-017-0775-3
ScloveS. L. (1987). Application of model-selection criteria to some problems in multivariate analysis. Psychometrika, 52(3), 333-343. https://doi.org/10.1007/BF02294360
46.
SteinleyD.McDonaldR. P. (2007). Examining factor score distributions to determine the nature of latent spaces. Multivariate Behavioral Research, 42(1), 133-156. https://doi.org/10.1080/00273170701341217
47.
TaylorC.AllenA.KilgusS. P.von der EmbseN. P.GarbaczS. A. (2019). Development and validation of a parent version of the Social, Academic, and Emotional Behavior Risk Screener (SAEBRS) in an elementary sample. Behavioral Disorders, 44(4), 205-214. https://doi.org/10.1177/0198742918798561
48.
TeinJ.CoxeS.ChamH. (2013). Statistical power to detect the correct number of classes in latent profile analysis. Structural Equation Modeling: A Multidisciplinary Journal, 20(4), 640-657. https://doi.org/10.1080/10705511.2013.824781
49.
van DulmenM. H. M.EgelandB. (2011). Analyzing multiple informant data on child and adolescent behavior problems: Predictive validity and comparison of aggregation procedures. International Journal of Behavioral Development, 35(1), 84-92. https://doi.org/10.1177/0165025410392112
50.
VermuntJ. K. (2010). Latent class modeling with covariates: Two improved three-step approaches. Political Analysis, 18(4), 450-469. https://doi.org/10.1093/pan/mpq025
51.
von der EmbseN.P.IaccarinoS.MankinA.KilgusS.MagenE. (2017). Development and validation of the Social, Academic, and Emotional Behavior Risk Screener—Student Rating Scale. Assessment for Effective Intervention, 42(3), 186-192. https://doi.org/10.1177/1534508416679410
52.
von der EmbseN. P.KimE. S.KilgusS.DedrickR.TanakaM.SanchezA. (2019). Multi-informant universal screening: Evaluation of rater, item, and construct variance using a trifactor model. Journal of School Psychology, 77(December), 52-66. https://doi.org/10.1016/j.jsp.2019.09.005
53.
von der EmbseN.KimE.JenkinsA.SanchezA.KilgusS.EklundK. (under review). Informant discrepancy within universal screening in predicting distal outcomes. Journal of Psychopathology and Behavioral Assessment.
54.
WangY.KimE.FerronJ. M.DedrickR. F.TanT. X.StarkS. (2020). Testing measurement invariance across unobserved groups: The role of covariates in factor mixture modeling. Educational and Psychological Measurement. Advance online publication. https://doi.org/10.1177/0013164420925122
55.
WangY.KimE.JooS.-H.ChunS.AlamriA.LeeP.StarkS. (2020). Reconsidering multilevel latent class models: Can level-2 latent classes affect item response probabilities?Journal of Experimental Education. Advance online publication. https://doi.org/10.1080/00220973.2020.1737913
56.
YangC. C. (2006). Evaluating latent class analysis models in qualitative phenotype identification. Computational Statistics & Data Analysis, 50(4), 1090-1104. https://doi.org/10.1016/j.csda.2004.11.004
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
