In mean and covariance structure analysis, the chi-square difference test is often applied to evaluate the number of factors, cross-group constraints, and other nested model comparisons. Let model Ma be the base model within which model Mb is nested. In practice, this test is commonly used to justify Mb even when Ma is misspecified. The authors study the behavior of the chi-square difference test in such a circumstance. Monte Carlo results indicate that a nonsignificant chi-square difference cannot be used to justify the constraints in Mb. They also show that when the base model is misspecified, the z test for the statistical significance of a parameter estimate can also be misleading. For specific models, the analysis further shows that the intercept and slope parameters in growth curve models can be estimated consistently even when the covariance structure is misspecified, but only in linear growth models. Similarly, with misspecified covariance structures, the mean parameters in multiple group models can be estimated consistently under null conditions.
Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107, 238-246.
2.
Bollen, K. A. (2002). Latent variables in psychology and the social sciences. Annual Review of Psychology, 53, 605-634.
3.
Brouwers, A., & Tomic, W. (2001). The factorial validity of scores on the teacher interpersonal self-efficiency scale. Educational and Psychological Measurement, 61, 433-445.
4.
Browne, M. W., & Arminger, G. (1995). Specification and estimation of mean and covariance structure models. In G. Arminger, C. C. Clogg, & M. E. Sobel (Eds.), Handbook of statistical modeling for the social and behavioral sciences (pp. 185-249). New York: Plenum.
5.
Buse, A. (1982). The likelihood ratio, Wald, and Lagrange multiplier tests: An expository note. American Statistician, 36, 153-157.
6.
Byrne, B. M., Shavelson, R. J., & Muthen, B. (1989). Testing for the equivalence of factorial covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466.
7.
Chou, C.-P., & Bentler, P. M. (1990). Model modification in covariance structure modeling: A comparison among likelihood ratio, Lagrange multiplier, and Wald tests. Multivariate Behavioral Research, 25, 115-136.
8.
Cole, D. A., Maxwell, S. E., Arvey, R., & Salas, E. (1993). Multivariate group comparisons of variable systems: MANOVA and structural equation modeling. Psychological Bulletin, 114, 174-184.
9.
Curran, P. J. (2000). A latent curve framework for the study of developmental trajectories in adolescent substance use. In J. R. Rose, L. Chassin, C. C. Presson, & S. J. Sherman (Eds.), Multivariate applications in substance use research: New methods for new questions (pp. 1-42). Mahwah, NJ: Lawrence Erlbaum.
10.
Drasgow, F., & Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70, 662-680.
11.
Duncan, T. E., Duncan, S. C., Strycker, L. A., Li, F., & Alpert, A. (1999). An introduction to latent variable growth curve modeling: Concepts, issues, and applications. Mahwah, NJ: Lawrence Erlbaum.
12.
Hancock, G. R., Lawrence, F. R., & Nevitt, J. (2000). Type I error and power of latent mean methods and MANOVA in factorially invariant and noninvariant latent variable systems. Structural Equation Modeling, 7, 534-556.
13.
Hong, S. H., Malik, M. L., & Lee, M.-K. (2003). Testing configural, metric, scalar, and latent mean invariance across genders in sociotropy and autonomy using a non-Western sample. Educational and Psychological Measurement, 63, 636-654.
14.
Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
15.
Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409-426.
16.
Jöreskog, K. G. (1978). Structural analysis of covariance and correlation matrices. Psychometrika, 43, 443-477.
17.
Kano, Y. (1986). Conditions on consistency of estimators in covariance structure model. Journal of the Japan Statistical Society, 16, 75-80.
18.
Kano, Y. (2001). Structural equation modeling for experimental data. In R. Cudeck, S. H. C. du Toit, & D. Sörbom (Eds.), Structural equation modeling: Present and future (pp. 381-402). Chicago: Scientific Software.
19.
Kaplan, D. (1989). Power of the likelihood ratio test in multiple group confirmatory factor analysis under partial measurement invariance. Educational and Psychological Measurement, 49, 579-586.
20.
Kaplan, D., & George, R. (1995). A study of the power associated with testing factor mean differences under violations of factorial invariance. Structural Equation Modeling, 2, 101-118.
21.
Larose, S., Guay, F., & Boivin, M. (2002). Attachment, social support, and loneliness in young adulthood: A test of two models. Personality and Social Psychology Bulletin, 28, 684-693.
22.
Lawley, D. N., & Maxwell, A. E. (1971). Factor analysis as a statistical method (2nd ed.). New York: American Elsevier.
23.
Lee, S. Y. (1985). On testing functional constraints in structural equation models. Biometrika, 72, 125-131.
24.
Lee, S. Y., & Leung, T. K. (1982). Covariance structure analysis in several populations. Psychometrika, 47, 297-308.
25.
MacCallum, R. (1986). Specification searches in covariance structure modeling. Psychological Bulletin, 100, 107-120.
26.
MacCallum, R. C., & Austin, J. T. (2000). Applications of structural equation modeling in psychological research. Annual Review of Psychology, 51, 201-226.
27.
Mardia, K. V., Kent, J. T., & Bibby, J. M. (1979). Multivariate analysis. New York: Academic Press.
28.
McArdle, J. J., & Epstein, D. (1987). Latent growth curves with developmental structure equation models. Child Development, 58, 110-133.
29.
Meredith, W., & Tisak, J. (1990). Latent curve analysis. Psychometrika, 55, 107-122.
30.
Satorra, A. (1989). Alternative test criteria in covariance structure analysis: A unified approach. Psychometrika, 54, 131-151.
31.
Satorra, A., & Saris, W. (1985). Power of the likelihood ratio test in covariance structure analysis. Psychometrika, 50, 83-90.
32.
Shapiro, A. (1984). A note on the consistency of estimators in the analysis of moment structures. British Journal of Mathematical and Statistical Psychology, 37, 84-88.
33.
Sörbom, D. (1974). A general method for studying differences in factor means and factor structures between groups. British Journal of Mathematical and Statistical Psychology, 27, 229-239.
34.
Steiger, J. H., Shapiro, A., & Browne, M. W. (1985). On the multivariate asymptotic distribution of sequential chi-square statistics. Psychometrika, 50, 253-264.
35.
Vispoel, W., Boo, J., & Bleiler, T. (2001). Computerized and paper-and-pencil versions of the Rosenberg Self-Esteem Scale: A comparison of psychometric features and respondent preferences. Educational and Psychological Measurement, 61, 461-474.
36.
Whiteside-Mansell, L., & Corwyn, R. F. (2003). Mean and covariance structure analysis: An examination of the Rosenberg Self-Esteem Scale among adolescents and adults. Educational and Psychological Measurement, 63, 163-173.
37.
Yuan, K.-H., & Bentler, P. M., (2004). Mean comparison: Manifest variable versus latent variable. Under review.
38.
Yuan, K.-H., Marshall, L.L.,& Bentler, P.M. (2003). Assessing the effect of model misspecifications on parameter estimates in structural equation models. Sociological Methodology, 33, 241-265.
