Relative importance of predictors: Comparison of Random Forests with Johnson's Relative Weights

Abstract

Decision makers in organizations frequently need to determine relative importance of multiple predictors of an important Dependent Variable. Using Multiple Regression for this purpose is often challenging when predictors are intercorrelated. In this paper we present the results of two Monte-Carlo studies comparing the effectiveness of two methods for determining relative importance of predictors under conditions of multicollinearity: Johnson’s Relative Weights (JRW) and Breiman’s Random Forests (RF). The following factors were systematically varied: number of predictors, correlations among predictors in population, regression model ${\mathit{R}}^2$ in population, number of observations per predictor, reliability of measurement, and standard deviation of regression coefficients across predictors in population. To serve as a benchmark measure of the relative importance of predictors, General Dominance Weights (GDW) method was used (also known as Shapley Value of the predictors in a regression), which defines predictor importance as an average increase in ${\mathit{R}}^2$ associated with that predictor across all possible regression submodels. Sample-based predictor importances determined by JRW and RF were compared to GDW importances in population. The implications of the results for practitioners are discussed.