The authors examined use of cross-validation in multiple regression research in organization studies by reviewing research published in three top journals in business policy and strategy (BPS) over 6 years. Application of a formula method of cross-validation to each study showed that shrinkage was often large. In some cases, shrinkage was greater than 50% of R2when sample sizes were less than 90. The authors recommend that organization studies researchers use formula cross-validation to improve the validity of inferences while avoiding the additional data collection and split samples required by empirical methods.
Get full access to this article
View all access options for this article.
References
1.
Bergh, D. D.
(1995). Problems with repeated measures analysis: Demonstration with a study of the diversification and performance relationship. Academy of Management Journal, 38(6), 1692-1708.
2.
Browne, M. W.
(1975). Predictive validity of a linear regression equation. British Journal of Mathematical and Statistical Psychology, 28, 79-87.
3.
Burket, G. R.
(1964). A study of reduced rank models for multiple prediction. Psychometric Monograph, No. 12.
4.
Cattin, P.
(1980). Estimation of the predictive power of a regression model. Journal of Applied Psychology, 65(4), 406-414.
5.
Claudy, J. C.
(1978). Multiple regression and validity estimation in one sample. Applied Psychological Measurement, 2(4), 595-607.
6.
Cook, T. D.
, & Campbell, D. T. (1976). The design and conduct of quasi-experiments and true experiments in field settings. In M. D. Dunnette (Ed.), Handbook of industrial and organizational psychology (pp. 223-326). Skokie, IL: Rand McNally.
7.
Cotter, K. L.
, & Raju, N. S. (1982). An evaluation of formula-adjusted population squared cross-validity estimates and factor score estimates in prediction. Educational and Psychological Measurement, 42, 493-519.
8.
Cureton, E. E.
(1951). Approximate linear restraints and best predictor weights. Educational and Psychological Measurement, 11, 12-29.
9.
Darlington, R. B.
(1968). Multiple regression in psychological research and practice. Psychological Bulletin, 69(3), 161-182.
10.
Drasgow, F.
, Dorans, N. J., & Tucker, L. R. (1979). Estimators of the squared cross-validity coefficient: A Monte Carlo investigation. Applied Psychological Measurement, 3(3), 387-399.
11.
Frank, R. E.
, Massy, W. F., & Morrison, D. G. (1965, August). Bias in multiple discriminant analysis. Journal of Marketing Research, 2, 250-258.
12.
Herzberg, P. A.
(1969). The parameters of cross-validation. Psychometric Monograph, No. 16.
13.
Hitt, M. A.
, Gimeno, J., & Hoskisson, R. E. (1998). Current and future research methods in strategic management. Organizational Research Methods, 1(1), 6-44.
14.
Huberty, C. J.
, & Mourad, S. A. (1980). Estimation in multiple correlation/prediction. Educational and Psychological Measurement, 40, 101-112.
15.
Larsson, R.
(1993). Case survey methodology: Quantitative analysis of patterns across case studies. Academy of Management Journal, 36(6), 1515-1546.
16.
MacMillan, I. C.
(1989). Delineating a forum for business policy scholars. Strategic Management Journal, 10(4), 391-395.
17.
MacMillan, I. C.
(1991). The emerging forum for business policy scholars. Strategic Management Journal, 12(2), 161-165.
18.
MacMillan, I. C.
, & Stern, I. (1987). Defining a forum for business policy scholars. Strategic Management Journal, 8(2), 183-186.
19.
Mitchell, T. R.
(1985). An evaluation of the validity of correlational research conducted in organizations. Academy of Management Review, 10(2), 192-205.
20.
Mosier, C. I.
(1951). Problems and designs of cross-validation. Educational and Psychological Measurement, 11, 5-11.
21.
Murphy, K. R.
(1984). Cost-benefit considerations in choosing among cross-validation. Personnel Psychology, 37, 15-22.
22.
Nayyar, P. R.
(1993). On the measurement of competitive strategy: Evidence from a large multi-product U.S. firm. Academy of Management Journal, 36(6), 1652-1669.
23.
Podsakoff, P. M.
, & Dalton, D. R. (1987). Research methodology in organiztion studies. Journal of Management, 13(2), 419-441.
24.
Rozeboom, W. W.
(1978). Estimation of cross-validated multiple correlation: A clarification. Psychological Bulletin, 85(6), 1348-1351.
25.
Rozeboom, W. W.
(1981). The cross-validation accuracy of sample regressions. Journal of Educational Statistics, 6(2), 179-198.
26.
Schmitt, N.
, Coyle, B. W., & Rauschenberger, J. (1977). A Monte Carlo evaluation of three formula estimates of cross-validated multiple correlation. Psychological Bulletin, 84(4), 751-758.
27.
Wherry, R. J.
(1931). A new formula for predicting the shrinkage of the coefficients of multiple correlation. Annals of Mathematical Statistics, 2, 440-457.
28.
Wiggins, R. R.
, & Ruefli, T. W. (1995). Necessary conditions for the predictive validity of strategic groups: Analysis without reliance on clustering techniques. Academy of Management Journal, 38(6), 1635-1656.
