ANCOVA and regression both exhibit a directional bias when measuring correlates of change. This bias confounds the comparison of changes between naturally occurring groups with large pretest differences (ANCOVA), or for identifying predictors of change when the predictor is correlated with pretest (regression). This bias is described in some detail. A computer simulation study is presented, which shows that properly identified structural equation models are not susceptible to this bias. Neither gain scores (posttest minus pretest) nor structural equation models exhibit the “regression bias.” Other factors, such as skewness, that may confound measurement of change are also discussed.
Get full access to this article
View all access options for this article.
References
1.
Browne, M. W. & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
2.
Darlington, R. B. (1990). Regression and linear models. New York: McGraw-Hill.
3.
Duncan, T. E. , Duncan, S. C., Alpert, A., Hops, H., Stoolmiller, M., & Muthen, B. (1997).Latent variable modeling of longitudinal and multilevel substance abuse data. Multivariate Behavioral Research, 32,275-318.
4.
Fan, X. , Thompson, B., & Wang, L. (1999).Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6,56-83.
5.
Holland, P. W. , & Rubin, D. B. (1983). On Lord’s Paradox. In H. Wainer & S. Messick (Eds.), Principles of modern psychological measurement (pp. 3-35). Hillsdale, NJ: Lawrence Erlbaum.
6.
Hu, L. , & Bentler, P. M. (1999).Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6,1-55.
7.
Huitema, B. E. (1980). The analysis of covariance and alternatives. New York: Wiley.
8.
Ip, C. , & Willson, V. L. (1998, April). Parameter estimates and fit indices in covariance structure analysis with nonnormal data. Paper presented at the 1998 Annual Meeting of the American Educational Research Association, San Diego, CA.
9.
Jamieson, J. (1993).The law of initial values: Five factors or two?International Journal of Psychophysiology, 14, 233-239.
10.
Jamieson, J. (1994). Correlates of reactivity: Problems with regression based methods. International Journal of Psychophysiology, 17,73-78.
11.
Jamieson, J. (1995).Measurement of change and the law of initial values: A computer simulation study. Educational and Psychological Measurement, 55,38-46.
12.
Jamieson, J. (1999).Dealing with baseline differences: Two principles and two dilemmas. International Journal of Psychophysiology, 31,155-161.
13.
Jamieson, J. , & Howk, S. (1992).The law of initial values: A four factor theory. International Journal of Psychophysiology, 12,53-61.
14.
Jöreskog, K. , & Sörbom, D. (1989). LISREL 7: A guide to the program and applications (2nd ed.). Chicago: SPSS.
15.
Lawrence, F. R. , & Hancock, G. R. (1998).Assessing change over time using latent growth modeling. Measurement and Evaluation in Counseling and Development, 30,211-224.
16.
Llabre, M. M. , Spitzer, S. S., Saab, P. G., Ironson, G. H., & Schneiderman, N. (1991).The reliability and specificity of delta versus residualized change as measures of cardiovascular reactivity to behavioral challenges. Psychophysiology, 28,701-711.
17.
Lord, F. E. (1967).A paradox in the interpretation of group comparisons. Psychological Bulletin, 68,304-305.
18.
MacCallum, R. C. , Kim, C., Malarkey, W. B., & Kiecolt-Glaser, J. K. (1997).Studying multivariate change using multilevel models and latent curve models. Multivariate Behavioral Research, 32,215-253.
19.
Maris, E. (1998).Covariance adjustment versus gain scores—revisited. Psychological Methods, 3,309-327.
20.
Raykov, T. (1992).Structural models for studying correlates and predictors of change. Australian Journal of Psychology, 44,101-112.
21.
Raykov, T. (1993).A structural equation model for measuring residualized change and discerning patterns of growth or decline. Applied Psychological Measurement, 17,53-71.
22.
Raykov, T. (1994).Studying correlates and predictors of longitudinal change using structural equation modeling. Applied Psychological Measurement, 18,63-77.
23.
Raykov, T. (1997).Simultaneous study of individual and group patterns of latent longitudinal change using structural equation modeling. Structural Equation Modeling, 4,212-236.
24.
Rogosa, D. R. (1988). Myths about longitudinal research. In K. W. Schaie, R. T. Campbell, W. Meredith, & S. C. Rawlings (Eds.), Methodological issues in aging research (pp. 171-209). New York: Springer.
25.
Rogosa, D. R. , & Willett, J. B. (1985).Understanding correlates of change by modeling individual differences in growth. Psychometrika, 50,203-228.
26.
SAS Institute . (1990a). SAS procedures guide (Version 6, 3rd ed.). Cary, NC: Author.
27.
SAS Institute . (1990b). SAS/STAT user’s guide (Version 6, 4th ed.). Cary, NC: Author.
28.
Schafer, W. D. (1992).Analysis of pretest-posttest designs. Measurement and Evaluation in Counselling and Development, 25,2-4.
29.
Shevlin, M. , & Miles, J.N.V. (1998).Effects of sample size, model specification and factor loadings on the GFI in confirmatory factor analysis. Personality and Individual Differences, 25,85-90.
30.
Steyer, R. , Eid, M., & Schwenkmezger, P. (1997). Modeling true intraindividual change: True change as a latent variable. Methods of Psychological Research [Online]. Available: www.hsp.de/MPR
31.
Wainer, H. (1991).Adjusting for differential base rates: Lord’s paradox again. Psychological Bulletin, 109,147-151.
32.
Willett, J. B. (1997). Measuring change: What individual growth modeling buys you. In E. Amsel & K. A. Reninger (Eds.), Change and development (pp. 213-243). Mahwah, NJ: Lawrence Erlbaum.
33.
Williams, R. H. , & Zimmerman, D. W. (1996).Are simple gain scores obsolete?Applied Psychological Measurement, 20, 59-69.
