The present Monte Carlo study assessed the effects of three potential confounding factors on structural equation modeling (SEM) fit indices and parameter estimates: data nonnormality, estimation method, and sample size. The major findings were that (a) relatively mild data nonnormality has little effect on SEM fit indices and parameter estimates; (b) under misspecified models, estimation method (maximum likelihood [ML] vs. generalized least squares [GLS]) has considerable influence on SEM incremental fit indices; and (c) some fit indices are more susceptible to the influence of sample size. Previous findings in the literature that SEM fit indices were consistent under different estimation methods may need to be revisited, because the finding was primarily based on Monte Carlo simulations involving true SEM models. Because SEM researchers rarely are certain whether they have correctly specified their models, it is critical that simulation studies are conducted in the presence of model misspecification, as in the present study.
Get full access to this article
View all access options for this article.
References
1.
Anderson,J. C. , & Gerbing, D. W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis. Psychometrika, 49, 155-173.
2.
Bagozzi, R. P. , Fomell, C., & Larcker, D. F. (1981). Canonical correlation analysis as a special case of structural relations model. Multivariate Behavioral Research, 16, 437-454.
3.
Baldwin, B. (1989). A primer in the use and interpretation of structural equation models. Measurement and Evaluation in Counseling and Development, 22, 100-112.
4.
Bearden, W. O. , Sharma, S., & Teel, J. R. (1982). Sample size effects on chi-square and other statistics used in evaluating causal models. Journal of Marketing Research, 19, 425-530.
5.
Bender, P. M. (1983). Some contributions to efficient statistics for structural models: Specification and estimation of moment structures. Psychometrika, 48, 493-517.
6.
Bender, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246.
7.
Bender, P. M. (1992). EQS structural equations program manual. Los Angeles: BMDP Statistical Software.
8.
Bender, P. M. (1994a). Foreword. In B. M. Byrne (Ed.), Structural equation modeling with EQS and EQS/Windows (p. vii). Thousand Oaks, CA: Sage.
9.
Bender, P. M. (1994b). On the quality of test statistics in covariance structure analysis: Caveat emptor. In C. R. Reynolds (Ed.), Cognitive assessment: A multidisciplinary perspective (pp. 237-260). New York: Plenum.
10.
Bender, P. M. , & Dudgeon, P. (1996). Covariance structure analysis: Statistical practice, theory, and directions. Annual Review of Psychology, 47, 563-592.
11.
Bollen, K. A. (1986). Sample size and Bender and Bonnett's nonnormed fit index. Psychometrika, 51, 375-377.
12.
Bollen, K. A. (1989). A new incremental fit index for general structural equation models. Sociological Methods & Research, 17, 303-316.
13.
Bollen, K. A. , & Long, J. S. (1993). Introduction. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 1-9). Newbury Park, CA: Sage.
14.
Bollen, K. A. , & Stine, R. (1992). Bootstrapping goodness-of-fit measures in structural equation models. Sociological Methods and Research, 21, 205-229.
15.
Boomsma, A. (1982). The robustness of LISREL against small sample sizes in factor analysis models. In K. G. Joreskog & H. Wold (Eds.), Systems under indirect observation: Causality, structure, prediction (Part I) (pp. 149-175). Amsterdam, The Netherlands: North-Holland.
16.
Boomsma, A. (1985). Nonconvergence, improper solutions, and starting values in LISREL maximum likelihood estimation. Psychometrika, 50, 229-242.
17.
Boomsma, A. (1987). The robustness of maximum likelihood estimation in structural equation models. In P. Cuttance & R. Ecob (Eds.), Structural modeling by example: Applications in educational, sociological, and behavioral research (pp. 160-188). New York: Cambridge University Press.
18.
Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures. British Journal of Mathematical and Statistical Psychology, 37, 62-83.
19.
Byrne, B. M. (1994). Structural equation modeling with EQS and EQSIU Windows: Basic concepts, applications, and programming. Newbury Park, CA: Sage.
20.
Byrne, B. M. (1998). Structural equation modeling with LISREL PRELIS and SIMPLIS: Basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum.
21.
Camstra, A. , & Boomsma, A. (1992). Cross-validation in regression and covariance structure analysis: An overview. Sociological Methods & Research, 21, 89-115.
22.
Chou, C. P. , Bentler, P. M., & Satorra, A. (1991). Scaled test statistics and robust standard errors for nonnormal data in covariance structure analysis: A Monte Carlo study. British Journal of Mathematical and Statistical Psychology, 44, 347-357.
23.
Cudeck, R. , & Henly, S. J. (1991). Model selection in covariance structure analysis and the "problem" of sample size: A clarification. Psychological Bulletin, 109, 512-519.
24.
Curran, P. J. , West, S. G., & Finch, J. F. (1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29.
25.
Fan, X. (1996). Structural equation modeling and canonical correlation analysis: What do they have in common?Structural Equation Modeling: A Multidisciplinary Journal, 4, 64-78.
26.
Fan, X. , Thompson, B. & Wang, L. (in press). The effects of sample size, estimation methods, and model misspecification on SEM fit indices. Structural Equation Modeling: A Multidisciplinary Journal.
27.
Fleishman, A. I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521-531.
28.
Gerbing, D. W. , & Anderson, J. C. (1985). The effects of sampling error and model characteristics on parameter estimation for maximum likelihood confirmatory factor analysis. Multivariate Behavioral Research, 20, 255-271.
29.
Gerbing, D. W. , & Anderson, J. C. (1993). Monte Carlo evaluations of goodness-of-fit indices for structural equation models. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 40-65). Newbury Park, CA: Sage.
30.
Hu, L. , Bentler, P. M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted?Psychological Bulletin, 112, 351-362.
31.
Ichikawa, M. , & Konishi, S. (1995). Application of the bootstrap methods in factor analysis. Psychometrika, 60, 77-93.
32.
James, L. R. , Mulaik, S. A., & Brett, J. M. (1982). Causal analysis: Models, assumptions, and data. Beverly Hills, CA: Sage.
33.
Jöreskog, K. G. , & Sorbom, D. (1989). LISREL 7: A guide to the program and applications (2nd ed.). Chicago: SPSS Inc.
34.
Jöreskog, K. G. , & Sorbom, D. (1993). LISREL 8 user's reference guide. Chicago: Scientific Software International, Inc.
35.
Kaiser, H. F. , & Dickman, K. (1962). Sample and population score matrices and sample correlation matrices from an arbitrary population correlation matrix. Psychometrika, 27, 179-182.
36.
Kano, Y. , Berkane, M., & Bentler, P. M. (1990). Covariance structure analysis with heterogeneous kurtosis parameters. Biometrika, 77, 575-585.
37.
LaDu, T. J. , & Tanaka, J. S. (1989). The influence of sample size, estimation method, and model specification on goodness-of-fit assessment in structural equation models. Journal of Applied Psychology, 74, 625-635.
38.
Loehlin, J. C. (1992). Latent variable models: An introduction to factor, path, and structural analysis. Hillsdale, NJ: Lawrence Erlbaum.
39.
MacCallum, R. C. , Roznowski, M., & Necowitz, L. B. (1992). Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin, 111, 490-504.
40.
Maiti, S. S. , & Mukhejee, B. N. (1991). Two new goodness-of-fit indices for covariance matrices with linear structure. British Journal of Mathematical and Statistical Psychology, 44, 153-180.
41.
Marsh, H. W. , Balla, J. R., & Hau, K. T. (1996). An evaluation of incremental fit indices: A clarification of mathematical and empirical properties. In G. A. Marcoulides & R. E. Schumacker (Eds.), Advanced structural equation modeling: Issues and techniques (pp. 315-353). Mahwah, NJ: Lawrence Erlbaum.
42.
Marsh, H. W. , Balla, J. R., & McDonald, R. P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103, 391-410.
43.
Marsh, H. W. , & Hau, K. (1998). Is parsimony always desirable? Response to Sivo and Willson, Hoyle, Markus, Mulaik, Tweedledee, Tweedledum, the Cheshire Cat, and others. Journal of Experimental Education, 66, 274-285.
44.
McDonald, R. P. (1989). An index of goodness-of-fit based on noncentrality. Journal of Classification, 6, 97-103.
45.
Micceri, T. (1989). The unicorn, the normal curve, and other improbable creatures. Psychological Bulletin, 105, 156-166.
46.
Mooijaart, A. (1985). Factor analysis for non-normal variables. Psychometrika, 50, 323-342.
47.
Mulaik, S. A. , James, L. R., Van Alstine, J., Bennett, N., Lind, S., & Stillwell, C. D. (1989). An evaluation of goodness of fit indices for structural equation models. Psychological Bulletin, 105, 430-445.
48.
Pedhazur, E. J. , & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Lawrence Erlbaum.
49.
Rhee, K. (1993). Alternative estimation methods instructuralequationmodelingwithLISREL-7: Effects of noncontinuity and nonnormality of variables with varying sample sizes (doctoral dissertation, University of Illinois, 1992). Dissertation Abstracts International, 53(7), B3578. (University Microfilms No. 92-36,576)
50.
SAS Institute, Inc. (1990). SAS/STAT user's guide, version 6 (Vol. 1, 4th ed.). Cary, NC: Author.
51.
SAS Institute, Inc. (1997). SAS/STAT software: Changes and enhancements through release 6.12. Cary, NC: Author.
52.
Sobel, M. E. , & Bohrnstedt, G. W. (1985). Use of null models in evaluating the fit of covariance structure models. In N. B. Tuma (Ed.), Sociological methodology 1985 (pp. 152-178). San Francisco: Jossey-Bass.
53.
Steiger, J. H. (1989). EzATH: A supplementary module for SYSTAT and SYSGRAPH. Evanston, IL: SYSTAT, Inc.
54.
Steiger, J. H. (1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research, 25, 173-180.
55.
Steiger, J. H. , & Lind, J. C. (1980, May). Statistically based tests for the number of common factors. Paper presented at the annual meeting of the Psychometric Society, Iowa City, IA.
56.
Sugawara, H. M. , & MacCallum, R. C. (1993). Effect of estimation method on incremental fit indexes for covariance structure models. Applied Psychological Measurement, 17, 365-377.
57.
Tanaka, J. S. (1987). "How big is big?": Sample size and goodness-of-fit in structural equation models with latent variables. Child Development, 58, 134-146.
58.
Tanaka, J. S. (1993). Multifaceted conceptions of fit in structural equation models. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 10-39). Newbury Park, CA: Sage.
59.
Tanaka, J. S. , & Huba, G. J. (1989). A fit index for covariance structure models under arbitrary GLS estimation. British Journal of Mathematical and Statistical Psychology, 42, 233-239.
60.
Thompson, B. (1996). AERA editorial policies regarding statistical significance testing: Three suggested reforms. Educational Researcher, 25(2), 26-30.
61.
[Invited address presented at the 1998 annual meeting of the American Educational Research Association,.]
62.
Thompson, B. , & Daniel, L. G. (1996). Factor analytic evidence for the construct validity of scores: An historical overview and some guidelines. Educational and Psychological Measurement, 56, 213-224.
63.
Vale, C. D. , & Maurelli, V. A. (1983). Simulating multivariate nonnormal distributions. Psychometrika, 48, 465-471.
64.
Wang, L. , Fan, X., & Willson, V. (1996). Effects of nonnormal data on parameter estimates and fit indices for a model with latent and manifest variables: An empirical study. Structural Equation Modeling: A Multidisciplinary Journal, 3, 228-247.
65.
Wheaton, D. E. , Muthen, B., Alwin, D. F., & Summers, G. F. (1977). Assessing reliability and stability in panel models. In D. R. Heise (Ed.), Sociological methodology 1977 (pp. 84-136). San Francisco: Jossey-Bass.
