Ridge regression has been suggested as a technique which can circumvent difficulties associated with ordinary least squares (OLS) regression. In particular, ridge regression was developed in order to reduce the mean square error of the parameter estimates. It is shown that ridge regression also offers an advantage over OLS estimation when a validity shrinkage criterion is considered. Systematic comparisons of cross-validated multiple correlations indicate that ridge estimation is superior to OLS estimation when (1) the predictors are multicollinear, (2) the number of predictors is large relative to the sample size, and (3) the population multiple correlation is relatively small in magnitude.
Get full access to this article
View all access options for this article.
References
1.
Browne, M. W.A comparison of single sample and cross-validation methods for estimating the mean squared error of prediction in multiple linear regression . British Journal of Mathematical and Statistical Psychology, 1975, 28, 112-124. (a)
2.
Browne, M. W.Predictive validity of a linear regression equation . British Journal of Mathematical and Statistical Psychology, 1975, 28, 79-87. (b) FADEN AND BOBKO
3.
Darlington, R. B.Multiple regression in psychological research and practice . Psychological Bulletin, 1968, 69, 161-182.
4.
Dempster, A. P.Alternatives to least squares in multiple regression. In Multivariate Statistical Inference, D. G. Kabe and R. G. Gupta (Eds.). New York: North Holland Publishing Company, 1973.
5.
Dempster, A. P. , Schatzoff, M., and Wermuth, N.A simulation study of alternatives to ordinary least squares. Journal of the American Statistical Association, 1977, 72, 77-91.
6.
Guilkey, D. K. and Murphy, J. L.Directed ridge regression techniques in cases of multicollinearity. Journal of the American Statistical Association, 1975, 70, 767-775.
7.
Gunst, R. F. and Mason, R. L.Biased estimation in regression: An evaluation using mean squared error. Journal of the American Statistical Association, 1977, 72, 616-628.
8.
Hoerl, A. E.Application of ridge analysis to regression problems . Chemical Engineering Progress, 1962, 58, 54-59.
9.
Hoerl, A. E. and Kennard, R. W.Ridge regression: Applications to non-orthogonal problems. Technometrics, 1970, 12, 69-82. (a)
10.
Hoerl, A. E. and Kennard, R. W.Ridge regression: Biased estimation for non-orthogonal problems. Technometrics, 1970, 12, 55-67. (b)
11.
Hoerl, A. E. and Kennard, R. W.Ridge regression: Iterative estimation of the biasing parameter. Communications in Statistics, 1976, 5, 77-88.
12.
Hoerl, A. E. , Kennard, R. W., and Baldwin, K. F.Ridge regression: Some simulations. Communications in Statistics, 1975, 4, 105-123.
13.
Lawless, J. F.Ridge and related estimation procedures: Theory and practice . Communications in Statistics, 1978, 47, 139-164.
14.
Lawless, J. F. and Wang, P.A simulation study of ridge and other regression estimators. Communications in Statistics, 1976, 5, 307-323.
15.
McDonald, G. C. and Galarneau, D. I.Monte-Carlo evaluation of some ridge-type estimators. Journal of the American Statistical Association, 1975, 70, 407-416.
16.
Marquardt, D. W. and Snee, R. D.Ridge regression in practice. The American Statistician, 1975, 29, 3-20.
17.
Morrison, D. F.Multivariate statistical methods. New York: McGraw-Hill, 1967.
18.
Schmidt, F. L.The relative efficiency of regression and simple unit predictor weights in applied differential psychology . Educational and Psychological Measurement, 1971, 31, 699-714.
19.
Theobald, C. M.Generalizations of mean square error applied to ridge regression . Journal of the Royal Statistical Society, 1974, 36, 103-106.
20.
Wherry, R. J.A new formula for predicting the shrinkage of the coefficient of multiple correlation . Annals of Mathematical Statistics, 1931, 2, 440-457.
21.
Wherry, R. J.Underprediction from overfitting: 45 years of shrinkage . Personnel Psychology, 1975, 28, 1-18.
