Fitting latent growth models with small sample sizes and non-normal missing data

Abstract

This study investigates the performance of robust maximum likelihood (ML) estimators when fitting and evaluating small sample latent growth models with non-normal missing data. Results showed that the robust ML methods could be used to account for non-normality even when the sample size is very small (e.g., N < 100). Among the robust ML estimators, “MLR” was the optimal choice, as it was found to be robust to both non-normality and missing data while also yielding more accurate standard error estimates and growth parameter coverage. However, the choice “MLMV” produced the most accurate p values for the χ² test statistic under conditions studied. Regarding the goodness of fit indices, as sample size decreased, all three fit indices studied (i.e., comparative fit index, root mean square error of approximation, and standardized root mean square residual) exhibited worse fit. When the sample size was very small (e.g., N < 60), the fit indices would imply that a proposed model fit poorly, when this might not be actually the case in the population.

Keywords

Latent growth models missing data non-normality small sample

Get full access to this article

View all access options for this article.

References

Asparouhov

Muthén

(2005). Multivariate statistical modeling with survey data. In Proceedings of the Federal Committee on Statistical Methodology (FCSM) Research Conference.

Asparouhov

Muthén

(2010). Simple second order chi-square correction scaled chi-square statistics (Technical appendix). Los Angeles, CA.

Bandalos

D. L.

Leite

(2013). The role of simulation in structural equation modeling. Structural equation modeling: A second course (2nd ed., pp. 625–666). Information Age.

Bauer

D. J.

Curran

P. J.

(2003). Distributional assumptions of growth mixture models: Implications for overextraction of latent trajectory classes. Psychological Methods, 8(3), 338–363.

Bentler

P. M.

(1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246.

Bollen

K. A.

(1989). Structural equations with latent variables. Wiley.

Bollen

K. A.

Curran

P. J.

(2006). Latent curve models: A structural equation perspective (Vol. 467). Wiley-Interscience.

Boomsma

(1982). The robustness of LISREL against small sample sizes in factor analysis models. In Jöreskog

K. G.

Wold

(Eds.), Systems under indirect observation: Causality, structure, prediction, (Part 1, pp. 149–173). Amsterdam: North-Holland.

Bradley

J. V.

(1978). Robustness? British Journal of Mathematical and Statistical Psychology, 31(2), 144–152.

10.

Brosseau-Liard

P. E.

Savalei

(2014). Adjusting incremental fit indices for nonnormality. Multivariate Behavioral Research, 49(5), 460–470.

11.

Brosseau-Liard

P. E.

Savalei

(2012). An investigation of the sample performance of two nonnormality corrections for RMSEA. Multivariate Behavioral Research, 47(6), 904–930.

12.

Chou

C. P.

Bentler

P. M.

Pentz

M. A.

(1998). Comparisons of two statistical approaches to study growth curves: The multilevel model and the latent curve analysis. Structural Equation Modeling: A Multidisciplinary Journal, 5(3), 247–266.

13.

Curran

P. J.

West

S. G.

Finch

J. F.

(1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1(1), 16.

14.

Enders

C. K.

(2010). Applied missing data analysis. Guilford Press.

15.

Ferguson

C. J.

(2009). An effect size primer: A guide for clinicians and researchers. Professional Psychology: Research and Practice, 40, 532–538.

16.

Ferrer

Balluerka

Widaman

K. F.

(2008). Factorial invariance and the specification of second-order latent growth models. Methodology: European Journal of Research Methods for the Behavioral and Social Sciences, 4(1), 22.

17.

Finney

DiStefano

(2013). Nonnormal and categorical data in structural equation modeling. In Hancock

G. R.

Mueller

R. O.

(Eds.), Structural equation modeling: A second course (2nd ed., pp. 439–492). Information Age.

18.

Foldnes

Olsson

U. H.

(2016). A simple simulation technique for nonnormal data with prespecified skewness, kurtosis, and covariance matrix. Multivariate Behavioral Research, 51(2-3), 207–219.

19.

Gao

Shi

Maydeu-Olivares

(2020). Estimating the maximum likelihood root mean square error of approximation (RMSEA) with non-normal data: A Monte-Carlo study. Structural Equation Modeling: A Multidisciplinary Journal, 27(2), 192–201.

20.

Grimm

K. J.

Ram

Estabrook

(2017). Growth modeling: Structural equation and multilevel modeling approaches. Guilford Publications.

21.

Hatton

D. D.

Sideris

Skinner

Mankowski

Bailey

D. B.

Jr Roberts

Mirrett

(2006). Autistic behavior in children with fragile X syndrome: Prevalence, stability, and the impact of FMRP. American Journal of Medical Genetics Part A, 140(17), 1804–1813.

22.

Hedeker

Gibbons

R. D.

(2006). Longitudinal data analysis. Wiley.

23.

Hoogland

J. J.

Boomsma

(1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods & Research, 26(3), 329–367.

24.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55.

25.

Jöreskog

K. G.

Sörbom

(1988). LISREL 7: A guide to the program and applications (2nd ed.). International Education Services.

26.

Maydeu-Olivares

(2017). Maximum likelihood estimation of structural equation models for continuous data: Standard errors and goodness of fit. Structural Equation Modeling: A Multidisciplinary Journal, 24(3), 383–394.

27.

McNeish

(2017). Brief research report: Growth models with small samples and missing data. The Journal of Experimental Education, 86(4), 1–12.

28.

McNeish

Harring

J. R.

(2017a). Correcting model fit criteria for small sample latent growth models with incomplete data. Educational and Psychological Measurement, 77(6), 990–1018.

29.

McNeish

Matta

(2018). Differentiating between mixed-effects and latent-curve approaches to growth modeling. Behavior Research Methods, 50(4), 1398–1414.

30.

McNeish

Stapleton

L. M.

(2016a). Modeling clustered data with very few clusters. Multivariate Behavioral Research, 51(4), 495–518.

31.

McNeish

D. M.

Harring

J. R.

(2017b). Clustered data with small sample sizes: Comparing the performance of model-based and design-based approaches. Communications in Statistics-Simulation and Computation, 46(2), 855–869.

32.

McNeish

D. M.

Stapleton

L. M.

(2016b). The effect of small sample size on two-level model estimates: A review and illustration. Educational Psychology Review, 28(2), 295–314.

33.

Meredith

Tisak

(1984, June). “Tuckerizing” curves. Paper presented at the meeting of the psychometric society, Santa Barbara, CA.

34.

Meredith

Tisak

(1990). Latent curve analysis. Psychometrika, 55(1), 107–122.

35.

Micceri

(1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105(1), 156–166.

36.

Moshagen

(2012). The model size effect in SEM: Inflated goodness-of-fit statistics are due to the size of the covariance matrix. Structural Equation Modeling: A Multidisciplinary Journal, 19(1), 86–98.

37.

Muthén

L. K.

Muthén

B. O.

(1998-2018). Mplus user’s guide (6th ed.). Muthén & Muthén.

38.

Muthén

L. K.

Muthén

B. O.

(2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9(4), 599–620.

39.

Nunnally

J. C.

Bernstein

I. H.

(1967). Psychometric theory (Vol. 226). McGraw-Hill.

40.

Olvera Astivia

O. L.

Zumbo

B. D.

(2015). A cautionary note on the use of the Vale and Maurelli method to generate multivariate, nonnormal data for simulation purposes. Educational and Psychological Measurement, 75(4), 541–567.

41.

Pavlov

Shi

Maydeu-Olivares

(2020). Chi-square difference tests for comparing nested models: An evaluation with non-normal data. Structural Equation Modeling: A Multidisciplinary Journal, 27(6), 908–917.

42.

Pornprasertmanit

Miller

Schoemann

A. M.

(2012 ). R packagesimsem: SIMulated structural equation modeling. Available from the Comprehensive R Archive Network: http://cranr-project.org.

43.

R Development Core Team. (2015). R: A Language and environment for statistical computing. Author.

44.

Ross

C. A.

(1997). Dissociative identity disorder: Diagnosis, clinical features, and treatment of multiple personality. John Wiley & Sons.

45.

Rubin

D. B.

(1976). Inference and missing data. Biometrika, 63(3), 581–592.

46.

Satorra

Bentler

P. M.

(1994). Corrections to test statistics and standard errors in covariance structure analysis. In Von Eye

Clogg

C. C.

(Eds.), Latent variable analysis. Applications for developmental research (pp. 399–419). SAGE.

47.

Savalei

(2010). Expected versus observed information in SEM with incomplete normal and nonnormal data. Psychological Methods, 15, 352–367.

48.

Savalei

(2014). Understanding robust corrections in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 21(1), 149–160.

49.

Savalei

(2018). On the computation of the RMSEA and CFI from the mean-and-variance corrected test statistic with nonnormal data in SEM. Multivariate Behavioral Research, 53(3), 419–429.

50.

Shi

Lee

Fairchild

A. J.

Maydeu-Olivares

(2019). Fitting ordinal factor analysis models with missing data: A comparison between pairwise deletion and multiple imputation. Educational and Psychological Measurement, 80(1), 41–66.

51.

Shi

Lee

Maydeu-Olivares

(2019). Understanding the model size effect on SEM fit indices. Educational and Psychological Measurement, 79(2), 310–334.

52.

Shi

Lee

Terry

R. A.

(2018). Revisiting the model size effect in structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 25(1), 21–40.

53.

Steiger

Lind

J. C.

(1980, May). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the annual spring meeting of the psychometric society, Iowa City.

54.

Vale

C. D.

Maurelli

V. A.

(1983). Simulating multivariate nonnormal distributions. Psychometrika, 48(3), 465–471.

55.

West

S. G.

Taylor

A. B.

(2009). Evaluating model fit for growth curve models: Integration of fit indices from SEM and MLM frameworks. Psychological Methods, 14(3), 183.

56.

Zheng

(2017). Latent growth curve analysis with item response data: Model specification, estimation, and panel attrition [Unpublished doctoral dissertation]. University of Maryland.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

2.07 MB