Latent growth models implemented in multilevel models (MLM) or structural equation models (SEM) may be used to analyze longitudinal data with an emphasis on interindividual and intraindividual differences. The main objective of this study is to compare methods of handling missing time-invariant data under the assumption of missing completely at random. Listwise deletion (LD), mean substitution (MS), the expectation-maximization (EM) algorithm, multiple imputation (MI), and full information maximum likelihood (FIML) are compared via a computer simulation study. The findings show that FIML and LD generally perform best, whereas the standard errors of EM and MI are usually underestimated. The results on MS are generally acceptable except when the percentage of missingness is large. Practical implications and directions for future research are discussed.
Biesanz, J.C., Deeb-Sossa, N., Aubrecht, A.M., Bollen, K.A., & Curran, P.J. (2004). The role of coding time in estimating and interpreting growth curve models. Psychological Methods, 9, 30-52.
5.
Bliese, P.D., & Ployhart, R.E. (2002). Growth modeling using random coefficients models: Model building, testing, and illustrations. Organizational Research Methods, 5, 362-387.
Bryant, A.L., Schulenberg, J.E., O'Malley, P.M., Bachman, J.G., & Johnston, L.D. (2003). How academic achievement, attitudes, and behaviors relate to the course of substance use during adolescence: A 6-year, multiwave national longitudinal study. Journal of Research on_Adolescence, 13, 361-397.
8.
Bryk, A.S., & Raudenbush, S.W. (1987). Application of hierarchical linear models to assessing change. Psychological Bulletin,101, 147-158.
9.
Bryk, A.S., & Raudenbush, S.W. (1989). Methodology for cross-level organizational research. Research inSociology of Organizations , 7, 233-273.
10.
Chan, D. (1998). Functional relations among constructs in the same content domain at different levels of analysis: A typology of composition models. Journal of Applied Psychology, 83, 234-246.
11.
Chan, D. (2002). The conceptualization and analysis of change over time: An integrative approach incorporating longitudinal mean and covariance structure analysis (LMACS) and multiple indicator latent growth modeling (MLGM). Organizational Research Methods, 1, 421-483.
12.
Chan, D., Ramey, S., Ramey, C., & Schmitt, N. (2000). Modeling intraindividual changes in children's social skills at home and at school: A multivariate latent growth approach to understanding between-settings differences in children's social skill development . Multivariate Behavioral Research, 35, 365-396.
13.
Cheong, J., MacKinnon, & D.P., Khoo, S.T. (2003). Investigation of mediational processes using parallel process latent growth curve modeling. Structural Equation Modeling, 10, 238-262.
14.
Cheung, M.W.L., & Au, K. (2005). Applications of multilevel structural equation modeling to cross-cultural research. Structural Equation Modeling , 12, 598-619.
15.
Cheung, M.W.L., Leung, K., & Au, K. (in press). Evaluating multilevel models in cross-cultural research: An illustration with social axioms. Journal of Cross-Cultural Psychology.
16.
Chou, C., Bentler, P.M., & Pentz, M.A. (1998). Comparisons of two statistical approaches to study growth curves: The multilevel model and the latent curve analysis. Structural Equation Modeling, 5, 247-266.
17.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
18.
Collins, L.M. (2006). Analysis of longitudinal data: The integration of theoretical model, temporal design, and statistical model. Annual Review ofPsychology, 57, 505-528.
19.
Collins, L.M., Schafer, J.L., & Kam, C. (2001). A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychological Methods, 6, 330-351.
20.
Curran, P.J. (2003). Have multilevel models been structural equation models all along?Multivariate Behavioral Research, 38, 529-569.
21.
Dempster, A.P., Laird, N.M., & Rubin, D.B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, Series B, 39, 1-38.
22.
Enders, C.K. (2001). A primer on maximum likelihood algorithms available for use with missing data. Structural Equation Modeling, 8, 128-141.
23.
Enders, C.K., & Peugh, J.L. (2004). Using an EM covariance matrix to estimate structural equation models with missing data: Choosing an adjusted sample size to improve the accuracy of inferences. Structural Equation Modeling, 11, 1-19.
24.
Fichman, M., & Cummings, J.N. (2003). Multiple imputation for missing data: Making the most of what you know. Organizational Research Methods, 6, 282-308.
25.
Fredriksen, K., Rhodes, J., Reddy, R., & Way, N. (2004). Sleepless in Chicago: Tracking the effects of adolescent sleep loss during the middle school years. Child Development, 75, 84-95.
26.
Gibson, N.M., & Olejnik, S. (2003). Treatment of missing data at the second level of hierarchical linear models. Educational and Psychological Measurement, 63, 204-238.
27.
Goldstein, H. (2003). Multilevel statistical models (3rd ed.). London: Arnold.
28.
Graham, J.W., Hofer, S.M., & MacKinnon, D.P. (1996). Maximizing the usefulness of data obtained with planned missing value patterns: An application of maximum likelihood procedures. Multivariate Behavioral Research, 31, 197-218.
29.
Gravetter, F.J., & Forzano, L.B. (2003). Research methods for the behavioral sciences. Belmont, CA: Thomson/Wadsworth .
30.
Hofmann, D.A. (1997). An overview of the logic and rationale of hierarchical linear models. Journal of Management, 23, 723-744.
31.
Hofmann, D.A., & Gavin, M.B. (1998). Centering decisions in hierarchical linear models: Implications for research in organizations. Journal of Management , 24, 623-641.
32.
Hoogland, J.J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods and Research, 26, 329-367.
33.
Hox, J.J. (2002). Multilevel analysis: Techniques and applications . Mahwah, NJ: Lawrence Erlbaum.
Huang, X., & Van de Vliert, E. (2003). Comparing work behaviors across cultures: A cross-level approach using multilevel modeling. International Journal of Cross Cultural Management, 3, 167-182.
36.
Jöreskog, K.G., & Sörbom, D. (1996). LISREL 8: A user's reference guide. Chicago: Scientific Software International.
37.
Klein, K.J., & Kozlowski, S.W.J. (2000). From micro to meso: Critical steps in conceptualizing and conducting multilevel research. Organizational Research Methods, 3, 211-236.
38.
Klein, K.J., Tosi, H., & Cannella, Jr., A.A. (1999). Multilevel theory building: Benefits, barriers, and new developments. Academy of Management Review, 24, 243-248.
39.
Kozlowski, S.W.J., & Klein, K.J. (2000). A multilevel approach to theory and research in organizations: Contextual, temporal, and emergent properties. In K. J. Klein, S.W.J. Kozlowski (Eds.), Multilevel theory, research and methods in organizations: Foundations, extensions, and new directions (pp. 3-90). San Francisco: Jossey-Bass.
40.
MacCallum, R.C., Kim, C., Malarkey, W.B., & Kiecolt-Galser, J.K. (1997). Studying multivariate change using multilevel models and latent curve models. Multivariate Behavioral Research, 32, 215-253.
41.
Mehta, P.D., & Neale, M.C. (2005). People are variables too: Multilevel structural equation modeling. Psychological Methods, 10, 259-284.
42.
Mehta, P.D., & West, S.G. (2000). Putting the individual back into individual growth curves. Psychological Methods, 5, 23-43.
43.
Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika , 55, 107-122.
Muthén, B.O., & Curran, P.J. (1997). General longitudinal modeling of individual differences in experimental designs: A latent variable framework for analysis and power estimation. Psychological Methods, 2, 371-402.
46.
Muthén, L.K., & Muthén, B.O. (2002). How to use a Monte Carlo study to decide on sample size and determine power. Structural Equation Modeling, 9, 599-620.
Newman, D.A. (2003). Longitudinal modeling with randomly and systematically missing data: A simulation of ad hoc, maximum likelihood, and multiple imputation techniques. Organizational Research Methods, 6, 328-362.
49.
Pinheiro, J.C., & Bates, D.M. (2000). Mixed-effects models in S and S-PLUS. New York: Springer.
50.
Ployhart, R.E., & Hakel, M.D. (1998). The substantive nature of performance variability: Predicting interindividual differences in intraindividual performance. Personnel Psychology, 51, 859-901.
51.
R Development Core Team. (2006). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN3-900051-07-0, URL: http://www.r-project.org
52.
Rao, C.R. (1958). Some statistical methods for comparison of growth curves. Biometrics, 14, 1-17.
53.
Raudenbush, S.W. (2001). Comparing personal trajectories and drawing causal inferences from longitudinal data. Annual Review ofPsychology , 52, 501-525.
54.
Raudenbush, S.W., & Bryk, A.S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.
55.
Raudenbush, S.W., & Willms, J. (1995). The estimation of school effects. Journal of Educational and Behavioral Statistics, 20, 1-41.
56.
Raykov, T. (2005). Analysis of longitudinal studies with missing data using covariance structure modeling with full-information maximum likelihood . Structural Equation Modeling, 12, 493-505.
57.
Roth, P.L. (1994). Missing data: A conceptual review for applied psychologists. Personnel Psychology, 47, 537-560.
58.
Rovine, M.J., & Molenaar, P.C.M. (1998). A nonstandard method for estimating a linear growth model in LISREL. International Journal of Behavioral Development, 222, 453-473.
59.
Rovine, M.J., & Molenaar, P.C.M. (2000). A structural modeling approach to a multilevel random coefficient model. Multivariate Behavioral Research, 35, 51-88.
60.
Rubin, D.B. (1987). Multiple imputation for nonresponse in surveys . New York: Wiley.
61.
Savalei, V., & Bentler, P.M. (2005). A statistically justified pairwise ML method for incomplete for incomplete nonnormal data: A comparison with direct ML and pairwise ADF. Structural Equation Modeling, 12, 183-214.
Schafer, J.L. (2002). Norm: Analysis of multivariate normal datasets with missing values. R package version 1.0-9. http://www.stat.psu.edu/~jls/misoftwa.html#aut
64.
Schafer, J.L., & Graham, J.W. (2002). Missing data: Our view of the state of the art . Psychological Methods, 7, 147-177.
65.
Schafer, J.L., & Olsen, M.K. (1998). Multiple imputation for multivariate missing-data problems: A data analyst's perspective. Multivariate Behavioral Research, 33, 545-571.
66.
Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. Boca Raton, FL: Chapman & Hall.
67.
Tucker, L.R. (1958). Determination of parameters of a functional relation by factor analysis. Psychometrika, 23, 19-23.
68.
Venables, W.N., & Ripley, B.D. (2002). Modern applied statistics with S (4th ed.). New York: Springer.
69.
Wendorf, C.A. (2002). Comparisons of structural equation modeling and hierarchical linear modeling approaches to couples' data. Structural Equation Modeling, 9, 126-140.
70.
Willett, J.B., & Sayer, A.G. (1994). Using covariance structure analysis to detect correlates and predictors of change. Psychological Bulletin,116, 363-381.
71.
Wothke, W. (1993). Nonpositive definite matrices in structural modeling . In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp.256-293). Newbury Park, CA: Sage.
72.
Wothke, W. (2000). Longitudinal and multi-group modeling with missing data. In T. D. Little, K. U. Schnabel, & J. Baumert (Eds.). Modeling longitudinal and multiple group data: Practical issues: Applied approaches and specific examples (pp. 219-240). Mahwah, NJ: Lawrence Erlbaum.
