Restricted accessResearch articleFirst published online 1999-06
Statistically Differentiating Between Interaction and Nonlinearity in Multiple Regression Analysis: A Monte Carlo Investigation of a Recommended Strategy
The procedure recommended by Lubinski and Humphreys for differentiating between moderated and nonlinear regression models is shown to evidence statistical problems that are characteristic of stepwise procedures. A Monte Carlo study was conducted in which random samples were generated from populations with known nonlinear or non-additive forms. Factors manipulated in the study included the correlation between regressors, regressor reliability, population R2, effect size of the nonlinear or nonadditive components, and sample size. Type I error was not adequately controlled under any condition examined and statistical power was poor in many conditions. Results were interpreted in terms of researchers’ need to differentiate between exploratory (stepwise regression) and confirmatory (theory testing) aspects of inquiry.
Get full access to this article
View all access options for this article.
References
1.
Aiken, L. , & West, S. (1991). Multiple regression. Newbury Park, CA: Sage.
2.
Alexander, R. A. , & DeShon, R. P. (1994). Effect of error variance heterogeneity on the power of tests for regression slope differences. Psychological Bulletin, 115, 308-314.
3.
Baron, R. M. , & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173-1182.
4.
Brush, S. G. (1974). Should the history of science be rated X?Science, 183, 1164-1172.
5.
Busemeyer, J. R. , & Jones, L. E. (1983). Analysis of multiplicative combination rules when the causal variables are measured with error. Psychological Bulletin, 93, 549-562.
6.
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.
7.
Copas, J. B. (1983). Regression, prediction and shrinkage. Journal of the Royal Statistical Society (Series B), 45, 311-354.
8.
Cronbach, L. J. (1987). Statistical tests for moderator variables: Flaws in analyses recently proposed. Psychological Bulletin, 102, 414-417.
9.
Dunlap, W. P. , & Kemery, E. R. (1987). Failure to detect moderating effects: Is multicollinearity the problem?Psychological Bulletin, 102, 418-420.
10.
Dunlap, W. P. , & Kemery, E. R. (1988). Effects of predictor intercorrelations and reliabilities on moderated multiple regression. Organizational Behavior and Human Decision Processes, 41, 248-258.
11.
Feucht, T. E. (1989). Estimating multiplicative regression terms in the presence of measurement error. Sociological Methods and Research, 17, 257-282.
12.
Findley, M. J. , & Cooper, H. M. (1983). Locus of control and academic achievement: A literature review. Journal of Personality and Social Psychology, 44, 419-427.
13.
Fisicaro, S. A. , & Tisak, J. (1994). A theoretical note on the stochastics of moderated multiple regression. Educational and Psychological Measurement, 54, 32-41.
14.
Heise, D. R. (1986). Estimating nonlinear models: Correcting for measurement error. Sociological Methods and Research, 14, 447-472.
15.
Jaccard, J. , & Wan, C. K. (1995). Measurement error in the analysis of interaction effects between continuous predictors using multiple regression: Multiple indicator and structural equation approaches. Psychological Bulletin, 117, 348-357.
16.
Kenny, D. A. , & Judd, C. M. (1984). Estimating the nonlinear and interactive effects of latent variables. Psychological Bulletin, 90, 201-210.
17.
Lubinski, D. , & Humphreys, L. G. (1990). Assessing spurious “moderator effects”: Illustrated substantively with the hypothesized (“synergistic”) relation between spatial and mathematical ability. Psychological Bulletin, 107, 385-393.
18.
MacCallum, R. C. , & Mar, C. M. (1995). Distinguishing between moderator and quadratic effects in multiple regression. Psychological Bulletin, 118, 405-421.
19.
Maxwell, S. E. , Delaney, H. D., & Dill, C. A. (1984). Another look at ANCOVA versus blocking. Psychological Bulletin, 95, 136-147.
20.
McClelland, G. H. , & Judd, C. M. (1993). Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 114, 376-389.
21.
Morris, J. H. , Sherman, J. D., & Mansfield, E. R. (1986). Failures to detect moderating effects with ordinary least squares—moderated multiple regression: Some reasons and a remedy. Psychological Bulletin, 99, 282-288.
22.
Perlin, L. I. , Menagham, E. G., Lieberman, M. A., & Mullan, J. T. (1981). The stress process. Journal of Health and Social Behavior, 22, 337-356.
23.
Robey, R. R. , & Barcikowski, R. S. (1992). Type I error and the number of iterations in Monte Carlo studies of robustness. British Journal of Mathematical and Statistical Psychology, 45, 283-288.
24.
Saunders, D. R. (1955). The “moderator variable” as a useful tool in prediction. Proceedings of the 1954 invitational conference on testing problems (pp. 54-58). Princeton, NJ: Educational Testing Service.
25.
Shepperd, J. A. (1991). Cautions in assessing spurious “moderator effects.”Psychological Bulletin, 110, 315-317.
26.
Steiger, J. H. (1980). Tests for comparing elements of a correlation matrix. Psychological Bulletin, 87, 245-251.
27.
Thompson, B. (1995). Stepwise regression and stepwise discriminant analysis need not apply here: A guidelines editorial. Educational and Psychological Measurement, 55, 525-534.
28.
Thompson, B. (1998, April). Five methodology errors in educational research: The pantheon of statistical significance and other faux pas. Invited address presented at the annual meeting of the American Educational Research Association, San Diego. (ERIC Document Reproduction Service No. ED 419 023)
29.
Tukey, J. W. (1980). We need both exploratory and confirmatory. American Statistician, 34, 23-25.
