An introduction to multivariate adaptive regression splines

Abstract

Multivariate Adaptive Regression Splines (MARS) is a method for flexible modelling of high dimensional data. The model takes the form of an expansion in product spline basis functions, where the number of basis functions as well as the parameters associated with each one (product degree and knot locations) are automatically determined by the data. This procedure is motivated by recursive partitioning (e.g. CART) and shares its ability to capture high order interactions. However, it has more power and flexibility to model relationships that are nearly additive or involve interactions in at most a few variables, and produces continuous models with continuous derivatives. In addition, the model can be represented in a form that separately identifies the additive contributions and those associated with different multivariable interactions.

This paper summarizes the basic MARS algorithm, as well as extensions for binary response, categorical predictors, nested variables and missing values. It presents tips on interpreting the output of the standard FORTRAN implementation of MARS, and provides an example of MARS applied to a set of clinical data.

Get full access to this article

View all access options for this article.

References

Garber AM , Olshen RA , Zhang H. , Venkatraman ES Predicting high-risk cholesterol levels. International Statistical Review 1994; 62: 203-28.

Morgan JN , Sonquist JA Problems in the analysis of survey data, and a proposal. Journal of the American Statistical Association 1963; 58: 415-34.

Breiman L. , Friedman JH , Olshen RA , Stone CJ Classification and regression trees. Belmont, CA: Wadsworth, 1984.

Friedman JH , Silverman BW Flexible parsimonious smoothing and additive modeling (with discussion). Technometrics 1989; 31: 3-39.

Hastie T. , Tibshirani R. Generalized additive models. London: Chapman and Hall, 1990.

Friedman JH Multivariate adaptive regression splines (with discussion). Annals of Statistics. 1991; 19: 1-141.

Friedman JH Estimating functions of mixed ordinal and categorical variables using adaptive splines. Department of Statistics, Stanford University, Technical Report No. LCS 108, 1991 .

de Boor C. A practical guide to splines. New York, NY: Springer-Verlag, 1978.

Shumaker LL Fitting surfaces to scattered data. In: Approximation theory III. In: Lorentz GG , Chui CK , Shumaker LL , eds. New York: Academic Press, 1976: 203-68.

10.

Shumaker LL On spaces of piecewise polynomials in two variables. In: Singh SP , Burry JH , Watson B , eds. Approximation theory and spline functions. Boston: D Reidel Publishing Co, 1984: 151-97.

11.

Eubank RL Spline smoothing and nonparametric regression. New York : Marcel Dekker, 1988.

12.

Wahba G. Spline models for observational data. Monograph: SIAM, CBMS-NSF Regional Conference Series in Applied Mathematics, Vol. 59, 1990.

13.

Parzen E. On estimation of a probability density function and mode. Annals of Mathematical Statistics. 1962; 33: 1065-76.

14.

Craven P. , Wahba G. Smoothing noisy data with spline functions. Estimating the correct degree of smoothing by the method of generalized cross-validation. Numerical Mathematics. 1979; 31: 317-403.

15.

Smith PL Curve fitting and modeling with splines using statistical variable selection techniques. Hampton, VA: NASA, Langley Research Center, 1982.

16.

Breiman L. , Peters S. Comparing automatic bivariate smoothers (A public service enterprise). Berkeley: Department of Statistics, University of California, Technical Report No. 161, 1988 .

17.

Bellman RE Adaptive control processes. Princeton, NJ: Princeton University Press, 1961.

18.

Breiman L. Submodel selection and evaluation in regression I. The x-fixed case and little bootstrap. Department of Statistics, University of California, Berkeley, Technical Report No. 169, 1989.

19.

Stone M. Cross-validatory choice and assessment of statistical predictors (with discussion) . Journal of the Royal Statistical Society 1974; B36: 111-47.

20.

Efron B. Estimating the error rate of a prediction rule: improvement on cross-validation . Journal of the American Statistical Association 1983; 78: 316-31.

21.

Cox DR Analysis of binary data. London: Chapman and Hall, 1970.

22.

Hastie T. , Tibshirani R. , Buja A. Flexible discriminant analysis by optimal scoring. AT&T Bell Laboratories, Technical Report, 1993.

23.

Kooperberg C. , Stone CJ , Troung YK Hazard regression. Journal of the American Statistical Association 1995; 90: 78-94.

24.

Roosen CB Visualization and exploration of high-dimensional functions using the functional ANOVA decomposition [Doctoral Dissertation]. Department of Statistics, Stanford University, 1995.

25.

Efron B. , Tibshirani R. An introduction to the bootstrap. London: Chapman and Hall, 1993.