A key advantage of the root mean square error of approximation (RMSEA) is that under certain assumptions, the sample estimate has a known sampling distribution that allows for the computation of confidence intervals. However, little is known about the finite sampling behaviors of this measure under violations of these ideal asymptotic conditions. This information is critical for developing optimal criteria for using the RMSEA to evaluate model fit in practice. Using data generated from a computer simulation study, the authors empirically tested a set of theoretically generated research hypotheses about the sampling characteristics of the RMSEA under conditions commonly encountered in applied social science research. The results suggest that both the sample estimates and confidence intervals are accurate for sample sizes of n = 200 and higher, but caution is warranted in the use of these measures at smaller sample sizes, at least for the types of models considered here.
Get full access to this article
View all access options for this article.
References
1.
Anderson, James C. and David W. Gerbing. 1984. “The Effect of Sampling Error on Convergence, Improper Solutions, and Goodness-of-Fit Indices for Maximum Likelihood Confirmatory Factor Analysis.” Psychometrika49:155-173.
2.
Bentler, Peter M. 1990. “Comparative Fit Indexes in Structural Models.” Psychological Bulletin107:238-246.
3.
Bentler, Peter M. 1995. EQS: Structural Equations Program Manual, Version 5.0. Los Angeles: BMDP Statistical Software.
4.
Bentler, Peter M. and Douglass G. Bonett. 1980. “Significance Tests and Goodness-of-Fit in the Analysis of Covariance Structures.” Psychological Bulletin88:588-606.
5.
Bollen, Kenneth A. 1989a. Structural Equations With Latent Variables. New York: John Wiley.
6.
Bollen, Kenneth A. 1989b. “A New Incremental Fit Index for General Structural Equation Models.” Sociological Methods & Research17:303-316.
7.
Bollen, Kenneth A. and J. Scott Long. 1993. “Introduction.” Pp. 1-9 in Testing Structural Equation Models, edited by K. A. Bollen and J. S. Long. Newbury Park, CA: Sage.
8.
Boomsma, Anne. 1983. On the Robustness of LISREL (Maximum Likelihood Estimation) Against Small Sample Size and Nonnormality. Amsterdam: Sociometric Research Foundation.
9.
Browne, Michael W. 1984. “Asymptotic Distribution Free Methods in the Analysis of Covariance Structures.” British Journal of Mathematical and Statistical Psychology37:127-141.
10.
Browne, Michael W. and Robert Cudeck. 1993. “Alternative Ways of Assessing Model Fit.” Pp. 136-162 in Testing Structural Equation Models, edited by K. A. Bollen and J. S. Long. Newbury Park, CA: Sage.
11.
Chen, Feinian, Kenneth A. Bollen, Pamela Paxton, Patrick J. Curran, and James Kirby. 2001. “Improper Solutions in Structural Equation Models: Causes, Consequences, and Strategies.” Sociological Methods and Research29:468-508.
12.
Curran, Patrick J., Kenneth A. Bollen, Pamela Paxton, James Kirby, and Feinian Chen. 2002. “The Noncentral Chi-Square Distribution in Misspecified Structural Equation Models: Finite Sample Results From a Monte Carlo Simulation.” Multivariate Behavioral Research37:1-36.
13.
Curran, Patrick J., Stephen G. West, and John F. Finch. 1996. “The Robustness of Test Statistics to Nonnormality and Specification Error in Confirmatory Factor Analysis.” Psychological Methods1:16-29.
14.
Fan, Xitao, B. Thompson, and L. Wang. 1999. “Effects of Sample Size, Estimation Methods, and Model Specification on Structural Equation Modeling Fit Indexes.” Structural Equation Modeling6:56-83.
15.
Gerbing, David W. and James C. Anderson. 1993. “Monte Carlo Evaluations of Goodness-of-Fit Indices for Structural Equation Models.” Pp. 40-65 in Testing Structural Equation Models, edited by K. A. Bollen and J. S. Long. Newbury Park, CA: Sage.
16.
Hammervold, Randi. 1998. “Properties of Goodness of Fit Statistics for Structural Equation Models.” Doctoral dissertation, NTNU Trondheim Norgest teknisknaturvitenskapelige Universitet.
17.
Hu, Litze and Peter M. Bentler. 1998. “Fit Indices in Covariance Structure Modeling: Sensitivity to Underparameterized Model Misspecification.” Psychological Methods3:424-453.
18.
Jöreskog, Karl G. 1993. “Testing Structural Equation Models.” Pp. 294-316 in Testing Structural Equation Models, edited by K. A. Bollen and J. S. Long. Newbury Park, CA: Sage.
19.
Kaplan, David. 1990. “Evaluating and Modifying Covariance Structure Models: A Review and Recommendation.” Multivariate Behavioral Research25:285-305.
20.
MacCallum, Robert C. 1990. “The Need for Alternative Measures of Fit in Covariance Structure Modeling.” Multivariate Behavioral Research25:157-162.
21.
MacCallum, Robert C., Michael W. Browne, and H. M. Sugawara. 1996. “Power Analysis and Determination of Sample Size for Covariance Structure Modeling.” Psychological Methods1:130-149.
22.
Mariano, Robert S. 1982. “Analytical Small-Sample Distribution Theory in Econometrics: The Simultaneous-Equations Case.” International Economic Review23:503-533.
23.
McDonald, Roderick P. 1989. “An Index of Goodness of Fit Based on Noncentrality.” Journal of Classification6:97-103.
24.
McDonald, Roderick P. and Herbert W. Marsh. 1990. “Choosing a Multivariate Model: Non-centrality and Goodness of Fit.” Psychological Bulletin107:247-255.
25.
Muthén, Bengt O. and David Kaplan. 1985. “A Comparison of Some Methodologies for the Factor Analysis of Non-Normal Likert Variables.” British Journal of Mathematical and Statistical Psychology38:171-189.
26.
Muthén, Bengt O. and David Kaplan. 1992. “A Comparison of Some Methodologies for the Factor Analysis of Non-Normal Likert Variables: A Note on the Size of the Model.” British Journal of Mathematical and Statistical Psychology45:19-30.
27.
Paxton, Pamela, Patrick J. Curran, Kenneth A. Bollen, James Kirby, and Feinian Chen. 2001. “Monte Carlo Experiments: Design and Implementation.” Structural Equation Modeling8:287-312.
28.
Rigdon, Edward E. 1998. “The Equal Correlation Baseline Model for Comparative Fit Assessment in Structural Equation Modeling.” Structural Equation Modeling5:63-77.
29.
SAS. 2000. SAS/STAT User’s Guide: Version 8. Cary, NC: SAS Institute, Inc.
30.
Satorra, Albert and Willem Saris. 1985. “The Power of the Likelihood Ratio Test in Covariance Structure Analysis.” Psychomotrika50:83-90.
31.
Steiger, James H. 1989. EzPath: A Supplementary Module for SYSTAT and SYGRAPH. Evanston, IL: SYSTAT.
32.
Steiger, James H. 2000. “Point Estimation, Hypothesis Testing, and Interval Estimation Using the RMSEA: Some Comments and a Reply to Hayduk and Glaser.” Structural Equation Modeling7:149-162.
33.
Steiger, James H. and John C. Lind. 1980. “Statistically Based Tests for the Number of Common Factors.” Paper presented at the annual meeting of the Psychometric Society, May, Iowa City, IA.
34.
Steiger, James H., Alan Shapiro, and Michael W. Browne. 1985. “On the Multivariate Asymptotic Distribution of Sequential Chi-Square Statistics.” Psychometrika50:253-263.
35.
Sugawara, H. M. and Robert C. MacCallum. 1993. “Effect of Estimation Method on Incremental Fit Indexes for Covariance Structure Models.” Applied Psychological Measurement17:365-377.
36.
Tanaka, Jeff S. 1993. “Multifaceted Conceptions of Fit in Structural Equation Models.” In Testing Structural Equation Models, edited K. A. Bollen and J. S. Long. Newbury Park, CA: Sage.
37.
Tucker, Ledyard R. and C. Lewis. 1973. “A Reliability Coefficient for Maximum Likelihood Factor Analysis.” Psychometrika38:1-10.
