The multiple imputation two-stage (MI2S) approach holds promise for evaluating the model fit of structural equation models for ordinal variables with multiply imputed data. However, previous studies only examined the performance of MI2S-based residual-based test statistics. This study extends previous research by examining the performance of two alternative test statistics: the mean-adjusted test statistic (TM) and the mean- and variance-adjusted test statistic (TMV). Our results showed that the MI2S-based TMV generally outperformed other test statistics examined in a wide range of conditions. The MI2S-based root mean square error of approximation also exhibited good performance. This article demonstrates the MI2S approach with an empirical data set and provides Mplus and R code for its implementation.
BradleyJ. V. (1978). Robustness?British Journal of Mathematical and Statistical Psychology, 31(2), 144–152.
5.
BrowneM. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37(1), 62–83.
6.
BrowneM. W.CudeckR. (1993). Alternative ways of assessing model fit. In BollenK. A.LongJ. S. (Eds.), Testing structural equation models (pp. 136–162). Sage.
7.
ChenP.-Y.WuW.BrandtH.JiaF. (2020). Addressing missing data in specification search in measurement invariance testing with Likert-type scale variables: A comparison of two approaches. Behavior Research Methods, 52, 2567–2587.
8.
ChenP.-Y.WuW.Garnier-VillarrealM.KiteB. A.JiaF. (2020). Testing measurement invariance with ordinal missing data: A comparison of estimators and missing data techniques. Multivariate Behavioral Research, 55(1), 87–101.
9.
ChungS.CaiL. (2019). Alternative multiple imputation inference for categorical structural equation modeling. Multivariate Behavioral Research, 54(3), 323–337.
10.
CollinsL. M.SchaferJ. L.KamC.-M. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6(4), 330–351.
11.
DiStefanoC.MorganG. B. (2014). A comparison of diagonal weighted least squares robust estimation techniques for ordinal data. Structural Equation Modeling, 21(3), 425–438.
12.
EndersC. K. (2022). Applied missing data analysis (2nd ed.). Guilford Press.
13.
EndersC. K. (2023). Missing data: An update on the state of the art. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000563
14.
EndersC. K.MansolfM. (2018). Assessing the fit of structural equation models with multiply imputed data. Psychological Methods, 23(1), 76–93.
15.
FloraD. B.CurranP. J. (2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466–491.
16.
ForeroC. G.Maydeu-OlivaresA. (2009). Estimation of IRT graded response models: Limited versus full information methods. Psychological Methods, 14(3), 275–299.
17.
ForeroC. G.Maydeu-OlivaresA.Gallardo-PujolD. (2009). Factor analysis with ordinal indicators: A Monte Carlo study comparing DWLS and ULS estimation. Structural Equation Modeling, 16(4), 625–641.
18.
GelmanA.CarlinJ. B.SternH. S.DunsonD. B.VehtariA.RubinD. B. (2013). Bayesian data analysis (3rd ed.). Chapman and Hall/CRC.
19.
JacksonD. L.GillaspyJ. A.Jr.Purc-StephensonR. (2009). Reporting practices in confirmatory factor analysis: An overview and some recommendations. Psychological Methods, 14(1), 6–23.
20.
JiaF. (2023). Pooling test statistics across multiply imputed datasets for nonnormal items. Behavior Research Methods, 56, 1229–1243. https://doi.org/10.3758/s13428-023-02088-3
21.
JiaF.WuW. (2019). Evaluating methods for handling missing ordinal data in structural equation modeling. Behavior Research Methods, 51(5), 2337–2355.
LaiK. (2020). Correct point estimator and confidence interval for RMSEA given categorical data. Structural Equation Modeling: A Multidisciplinary Journal, 27(5), 678–695.
25.
LeeT.CaiL. (2012). Alternative multiple imputation inference for mean and covariance structure modeling. Journal of Educational and Behavioral Statistics, 37(6), 675–702.
26.
LiC.-H. (2016). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. Psychological Methods, 21(3), 369–387.
27.
LiK. H.RaghunathanT. E.RubinD. B. (1991). Large sample significance levels from multiply-imputed data using moment-based statistics and an F reference distribution. Journal of the American Statistical Association, 86, 1065–1073.
28.
LittleR. J. A.RubinD. B. (2020). Statistical analysis with missing data (3rd ed.). Wiley.
29.
LiuY.MillsapR. E.WestS. G.TeinJ.-Y.TanakaR.GrimmK. J. (2017). Testing measurement invariance in longitudinal data with ordered-categorical measures. Psychological Methods, 22(3), 486–506.
30.
LiuY.SriutaisukS. (2020). Evaluation of model fit in structural equation models with ordinal missing data: An examination of the D2 method. Structural Equation Modeling: A Multidisciplinary Journal, 27(4), 561–583.
31.
LiuY.SriutaisukS.ChungS. (2021). Evaluation of model fit in structural equation models with ordinal missing data: A comparison of the D2 and MI2S methods. Structural Equation Modeling: A Multidisciplinary Journal, 28(5), 740–762.
32.
MansolfM.JorgensenT. D.EndersC. K. (2020). A multiple imputation score test for model modification in structural equation models. Psychological Methods, 25(4), 393–411.
33.
McKelveyR. D.ZavoinaW. (1975). A statistical model for the analysis of ordinal level dependent variables. The Journal of Mathematical Sociology, 4(1), 103–120.
34.
MengX.-L.RubinD. B. (1992). Performing likelihood ratio tests with multiply-imputed data sets. Biometrika, 79, 103–111.
35.
MroczekD. K.KolarzC. M. (1998). The effect of age on positive and negative affect: A developmental perspective on happiness. Journal of Personality and Social Psychology, 75(5), 1333–1349.
36.
MuthénL. K.MuthénB. O. (1998–2017). Mplus user’s guide (8th ed.).
37.
OlssonU. (1979). Maximum likelihood estimation of the polychoric correlation coefficient. Psychometrika, 44(4), 443–460.
38.
RhemtullaM.Brosseau-LiardP. É.SavaleiV. (2012). When can categorical variables be treated as continuous? A comparison of robust continuous and categorical SEM estimation methods under suboptimal conditions. Psychological Methods, 17(3), 354–373.
39.
RosseelY. (2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36.
40.
RubinD. B. (1987). Multiple imputation for nonresponse in surveys. Wiley.
41.
SatorraA.BentlerP. M. (1994). Corrections to test statistics and standard errors in covariance structure analysis. In von EyeA.CloggC. C. (Eds.), Latent variables analysis: Applications for developmental research (pp. 399–419). Sage.
SavaleiV.RhemtullaM. (2013). The performance of robust test statistics with categorical data. British Journal of Mathematical and Statistical Psychology, 66(2), 201–223.
44.
SchaferJ. L.GrahamJ. W. (2002). Missing data: Our view of the state of the art. Psychological Methods, 7(2), 147–177.
45.
ShiD.DiStefanoC.McDanielH. L.JiangZ. (2018). Examining chi-square test statistics under conditions of large model size and ordinal data. Structural Equation Modeling: A Multidisciplinary Journal, 25(6), 924–945.
46.
ShiD.LeeT.FairchildA. J.Maydeu-OlivaresA. (2020). Fitting ordinal factor analysis models with missing data: A comparison between pairwise deletion and multiple imputation. Educational and Psychological Measurement, 80(1), 41–66.
47.
ShiD.ZhangB.LiuR.JiangZ. (2023). Evaluating close fit in ordinal factor analysis models with multiply imputed data. Educational and Psychological Measurement, 84, 171–189. https://doi.org/10.1177/001316442311588
48.
SongJ.RadlerB. T.LachmanM. E.MailickM. R.SiY.RyffC. D. (2021). Who returns? Understanding varieties of longitudinal participation in MIDUS. Journal of Aging and Health, 33(10), 896–907.
49.
TemanE. D. (2012). The performance of multiple imputation and full information maximum likelihood for missing ordinal data in structural equation models (Publication No. 3555133) [Doctoral dissertation, University of Northern Colorado]. ProQuest Dissertations and Theses Global.
50.
Van BuurenS. (2007). Multiple imputation of discrete and continuous data by fully conditional specification. Statistical Methods in Medical Research, 16(3), 219–242.
51.
WirthR. J.EdwardsM. C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12(1), 58–79.
52.
XiaY.YangY. (2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428.
53.
Yang-WallentinF.JöreskogK. G.LuoH. (2010). Confirmatory factor analysis of ordinal variables with misspecified models. Structural Equation Modeling, 17(3), 392–423.
54.
YuanK. H.BentlerP. M. (1998). Normal theory based test statistics in structural equation modelling. British Journal of Mathematical and Statistical Psychology, 51(2), 289–309.
55.
ZyphurM. J.BonnerC. V.TayL. (2023). Structural equation modeling in organizational research: The state of our science and some proposals for its future. Annual Review of Organizational Psychology and Organizational Behavior, 10, 495–517.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
