Multicollinearity is a phenomenon in which two or more identified predictor variables in a multiple regression model are co-dependent or highly correlated. The presence of this phenomenon can have a negative impact on the analysis as a whole and can severely limit the conclusions of the research study. This paper reviews and provides examples of the different ways in which multicollinearity can affect a research project, how to detect multicollinearity, and how one can reduce its impact through Ridge Regression.
Allison,P. (2012, September 10). When Can You Safely Ignore Multicollinearity? Retrieved from http://statisticalhorizons.com/multicollinearity.
2.
Afshartous,D., & Preston,R.A. (2011). Key Results of Interaction Models With Centering. Journal of Statistics Education, 19(3). Retrieved from https://www.amstat.org/publications/jse/v19n3/afshartous.pdf.
3.
Chatterjee,S.Hadi,A.S., & Price,B. (2000). Regression Analysis by Examples. (3rd ed.), New York, NY: Wiley VCH.
4.
Cross Validated. (2015, November 28). What is elastic net regularization, and how does it solve the drawbacks of Ridge (L2) and Lasso (L1)? Retrieved from https://stats.stackexchange.com/questions/184029/what-is-elastic-net-regularization-and-how-does-it-solve-the-drawbacks-of-ridge/184031#184031.
5.
Differences between L1 and L2 as Loss Function and Regularization. (2013, December 18). Retrieved from http://www.chioka.in/differences-between-l1-and-l2-as-loss-function-and-regularization/.
6.
Dixon,P.M. (1993). The bootstrap and the jackknife: Describing the precision of ecological indices. Design and Analysis of Ecological Experiments. 290-318, New York, NY: Chapman & Hall.
7.
Draper,N.R., & Smith,H. (2003). Applied regression analysis. (3 ed.), New York, NY: Wiley.
8.
Dorugade,A.V., & Kashid,D.N. (2010). Alternative Method for Choosing Ridge Parameter for Regression. Applied Mathematical Sciences, 4(9): 447-456.
9.
Efron,B., & Tibshirani,R.J. (1993). An Introduction to the Bootstrap. New York, NY: Chapman & Hall.
10.
Hair,J.F. Jr.Anderson,R.E.Tatham,R.L., & Black,W.C. (1995). Multivariate Data Analysis. (3rd ed). New York, NY: Macmillan.
11.
Hall,P. (1992). The Bootstrap and Edgeworth Expansion. New York, NY: Springer-Verlag.
Joshi,H.Kulkarni,H., & Deshpande,S. (2012). Multicollinearity Diagnostics in Statistical Modeling & Remedies to Deal With it Using SAS. Proceedings from PhUSE 2012. Retrieved from https://www.lexjansen.com/phuse/2012/sp/SP07.pdf.
14.
Kennedy,P. (1992). A Guide to Econometrics. Oxford: Blackwell.
15.
Marquardt,D.W. (1970). Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation. Technometrics, 12, 591-256.
16.
Montgomery,D.C.Peck,E.A., & Vining,G.G. (2001). Introduction to linear regression analysis. (3 ed.), New York, NY: Wiley.
Ng,A.Y. (2004). Feature selection, L1 vs L2 regularization, and rotational invariance. Proceedings of the 21 International Conference on Machine Learning. https://guides.library.uwa.edu.au/c.php?g=324904&p=2809543.
19.
Rosenberg,D. (2015). L1 and L2 Regularization [Lecture notes]. Retrieved from https://davidrosenberg.github.io/ml2015/docs/2b.L1L2-regularization.pdf.
20.
Semantic Community. (2013, December 29). SAS Public Data Sets. Retrieved from http://semanticommunity.info/Data_Science/SAS_Public_Data_Sets#Story.
21.
Shao,J., & Tu,D. (1995). The Jackknife and Bootstrap. New York, NY: Springer-Verlag.
22.
Shtatland,E.S.Kleinman,K., & Cain,E.M. (2004). A New Strategy of Model Building in PROC LOGISTIC With Automatic Variable Selection, Validation, Shrinkage and Model Averaging. Conference proceedings from SAS Users Group International Meeting 2004 (SUGI 29). Montreal, Canada. Retrieved from http://www2.sas.com/proceedings/sugi29/191-29.pdf.
23.
Stine,R. (1990). An introduction to bootstrap methods: Examples and ideas. Sociological Methods & Research, 18, 243-291.
24.
Unknown. (2010, December 3). Sample 24982: Jackknife and Bootstrap Analyses. Retrieved from http://support.sas.com/kb/24/982.html#pur.
25.
Unknown. (2018). What is Multicollinearity? [Lecture notes]. Retrieved from https://onlinecourses.science.psu.edu/stat501/node/344.
26.
van Wieringen,W.N. (2018). Ridge Regression [Lecture notes]. Retrieved from https://arxiv.org/pdf/1509.09169.pdf.
27.
Wicklin,R. (2013, march 20). Understanding Ridge Regression in SAS. Retrieved from http://blogs.sas.com/content/iml/2013/03/20/compute-ridge-regression.html.
