Several recent articles have suggested that ridge regression may provide an "optimal" procedure for dealing with the problems created by highly collinear regressors in linear models. In this article, we review the consequences of collinearity
among the regressors in a well-specified structural equation model and the several
variants of ridge regression that may be considered as possible responses to such
collinearity. Based on thts review and on application of several ridge regression
methods to an actual structural equation model in which collinearity is high, it
becomes clear that one or another form of investigator judgment is unavoidable
when any specific estimates are obtained via a ridge adjustment. The article indicates how the several ridge techniques implicitly involve distinct criteria whereby
an estimator should be judged and some of the complications involved in each.
Get full access to this article
View all access options for this article.
References
1.
Alexander, K.L. and B.K. Eckland ( 1975) "Contextual effects in the high school attainment process." Amer. Soc. Rev.40: 402-416.
2.
Alexander, K.L. and E.L. McDill (1976) "Selection and allocation within schools: some causes and consequences of curriculum placement." Amer. Soc. Rev.41: 963-980.
3.
Alexander, K.L., R.J. D'Amico, J. Fennessey, and E.L. McDill (1978) "Status composition and educational goals: an attempt at clarification." Reports of the Center for Social Organization of Schools 244, Johns Hopkins University, Baltimore, February.
4.
Campbell, E.Q. and C.N. Alexander (1965) "Structural effects and interpersonal relationships ." Amer. J. of Sociology71: 284-289.
5.
Carnahan, B. and J.O. Wilkes (1973) Digital Computing and Numencal Methods. New York: John Wiley.
6.
Conniffe, D. and J. Stone (1973) "A critical view of ridge regression." Statistician22: 181-187.
7.
Daniel, C. and F.S. Wood (1971) Fitting Equations to Data. New York: John Wiley.
8.
Deegan, J. (1975) "The process of political development: an illustrative use of a strategy for regression in the presence of multicollinearity." Soc. Methods and Research3: 384-415.
9.
Dempster, A.P., M. Schatzoff, and N. Wermuth (1977) "A simulation study of alternatives to ordinary least squares." J. of Amer. Statistical Assn.72: 77-106.
10.
Divgi, D.R. (1978) "A study of biased estimators in multiple regression with random predictors." Presented at the annual meetings of the American Educational Research Association, Toronto, March.
11.
Draper, N.R. and H. Smith (1966) Applied Regression Analysis. New York: John Wiley.
12.
Duncan, O.D., A.O. Haller, and A. Portes (1968) "Peer influences on aspirations : a reinterpretation ." Amer. J. of Sociology74: 119-137.
13.
Farrar, D.E. and R.R. Glauber (1967) "Multicollinearity in regression analysis: the problem revisited." Rev. of Economics and Statistics49: 92-107.
14.
Fennessey, J. (1977) "Improving inference for social research and social policy: the Bayesian paradigm." Social Sci. Research6: 309-327.
15.
Gunst, R.F. and R.L. Mason (1977) "Biased estimation in regression: an evaluation using mean squared error." J. of Amer. Statistical Assn.72: 616-628.
16.
Haitovsky, Y. (1969) "Multicollineanty in regression analysis: comment ." Rev. of Economics and Statistics59: 486-489.
17.
Henry, N.W. (1976) "A note on ridge regression." Soc. Methods and Research4: 495-500.
18.
Heyns, B. (1974) "Social selection and stratification within schools ." Amer. J. of Sociology79: 1434-1451
19.
Hocking, R.R. (1976) "The analysis and selection of variables in linear regression." Biometrics32: 1-49.
20.
Hoerl, A.E. and R.W. Kennard (forthcoming) "Ridge regression—1978." Technometrics .
——— (1970a) "Ridge regression: biased estimation for nonorthogonal problems." Technometrics1255-67.
23.
——— (1968) "Ridge regression: biased estimation for nonorthogonal problems." E. I. Dupont De Nemours & Co. technical report 13183, Wilmington, Delaware, June.
24.
Hoerl, A.E., R.W. Kennard, and K.F. Baldwin ( 1975) "Ridge regression: some simulations." Communications in Statistics4: 105-123.
25.
Hsiang, T.C. (1975) "A Bayesian view of ridge regression." Statistician24: 267-268.
26.
James, W. and C. Stein (1961) "Estimation with quadratic loss." Proceedings of the Fourth Berkeley Symposium in Mathematical Statistics and Probability 1: 361-379.
27.
Joreskog, K.G. and D. Sorbom (1976) LISREL III: Estimation of Linear Structural Equation Systems by Maximum Likelihood Methods: A FORTRAN IV Program. Chicago: National Educational Resources.
28.
Kasarda, J.D. and W.P. Shih ( 1977) "Optimal bias in ridge regression approaches to multicollinearity." Soc. Methods and Research5: 461-470.
29.
Lawless, J.F. and P. Wang (1976) "A simulation study of ridge and other regression estimators." Communications in Statistics: Theory and MethodsA5: 307-323.
30.
Lindley, D.V. and A.F.M. Smith (1972) "Bayes estimates for the linear model." J. of Royal Statistical Society, series B 34: 1-72.
31.
Marquardt, D.W. (1970) "Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation." Technometrics12: 591-612.
32.
Mason, R. and W.G. Brown (1975) "Multicollineanty problems and ridge regression in sociological research." Social Sci. Research4: 135-149.
33.
Mason, R.L., R.F. Gunst, and J.T. Webster (1975) "Regression analysis and problems of multicollinearity . Communications in Statistics4: 277-292.
34.
McDill, E.L. and L.C. Rigsby (1973) Structure and Process in Secondary Schools. Baltimore: Johns Hopkins Press.
35.
McDill, E.L., E.D. Meyers, Jr. and L.C. Rigsby (1967) "Institutional effects on the academic behavior of high school students." Sociology of Education40: 181-189.
36.
McDill, E.L., L.C. Rigsby, and E.D. Meyers, Jr. (1969) "Educational climates of high schools: their effects and sources." Amer. J. of Sociology74: 567-586.
37.
Rockwell, R.C. (1975) "Assessment of multicollinearity: the Haitovsky test of the determinant." Soc. Methods and Research3: 308-320.
38.
Shih, W.P. and S. Shih (1977) "Application of Monte Carlo simulation to ndge regression." Proceedings of the Social Statistics Section of the American Statistical Association: 1022-1027.
39.
Silvey, S.O. (1969) "Multicollinearity and imprecise information." J. of Royal Statistical Society, series B 31: 539-552.
40.
Thisted, R.A. (1976) "Ridge regression, minimax estimation, and empirical Bayes methods." Ph.D. dissertation, Stanford University .
41.
Vinod, H.D. (1978) " A survey of ridge regression and related techniques for improvements over ordinary least squares." Rev. of Economics and Statistics60: 121-131.
