Sage Journals: Discover world-class research

Abstract

In sparse penalized regressions, candidate covariates of different units need to be standardized beforehand so that the coefficient sizes are directly comparable and reflect their relative impacts, which leads to fairer variable selection. However, when covariates of mixed data types (e.g. continuous, binary or categorical) exist in the same dataset, the commonly used standardization methods may lead to different selection probabilities even when the covariates have the same impact on or level of association with the outcome. In this paper, we propose a novel standardization method that targets at generating comparable selection probabilities in sparse penalized regressions for continuous, binary or categorical covariates with the same impact. We illustrate the advantages of the proposed method in simulation studies, and apply it to the National Ambulatory Medical Care Survey data to select factors related to the opioid prescription in the US.

Keywords

Standardization dichotomization variable selection sparse penalized regression categorical covariates continuous covariates

Get full access to this article

View all access options for this article.

References

Friedman

Hastie

Tibshirani

. Regularization paths for generalized linear models via coordinate descent. J Stat Softw 2010; 33: 1–22.

Breheny

Huang

. Coordinate descent algorithms for nonconvex penalized regression, with applications to biological feature selection. Ann Appl Stat 2011; 5: 232–253.

Suarez-Alvarez

Pham

Prostov

et al. Statistical approach to normalization of feature vectors and clustering of mixed datasets. Proc R Soc A 2012; 468: 2630–2651.

Gelman

. Scaling regression inputs by dividing by two standard deviations. Stat Med 2008; 27: 2865–2873.

Bring

. How to standardize regression coefficients. Am Stat 1994; 3: 209–213.

Nair

Hinton

. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th International Conference on International Conference on Machine Learning. ICML’10, Madison, WI, USA: Omnipress. ISBN 9781605589077, pp. 807–814.

Baker

Tang

Allen

. Feature selection for data integration with mixed multiview data. Ann Appl Stat 2020; 14: 1676–1698.

Royston

Altman

Sauerbrei

. Dichotomizing continuous predictors in multiple regression: a bad idea. Stat Med 2006; 25: 127–141.

MacCallum

Zhang

Preacher

et al. On the practice of dichotomization of quantitative variables. Psychol Methods 2002; 7: 19–40.

10.

Harrell

. Categorizing continuous variables. https://discourse.datamethods.org/t/categorizing-continuous-variables/3.

11.

Centers for disease control and prevention. National ambulatory medical care survey data. https://www.cdc.gov/nchs/ahcd/index.htm (2016, accessed 01 September 2020).

12.

Tibshirani

. Regression shrinkage and selection via the lasso. J R Stat Soc Series B Stat Methodol 1996; 58: 267–288.

13.

Hastie

Tibshirani

Friedman

. The elements of statistical learning. 2nd ed. New York: Springer New York Inc., 2009. pp. 61–73.

14.

Yuan

Lin

. Model selection and estimation in regression with grouped variables. J R Stat Soc Series B Stat Methodol 2006; 68: 49–67.

15.

Meier

Van De Geer

Bühlmann

. The group lasso for logistic regression. J R Stat Soc Series B Stat Methodol 2008; 70: 53–71.

16.

Yang

Zou

. A fast unified algorithm for solving group-lasso penalized learning problems. Stat Comput 2015; 25: 1129–1141.

17.

Florentina

. Honest variable selection in linear and logistic regression models via

l 1

and

l 1 + l 2

penalization. Electron J Stat 2008; 2: 1153–1194.

18.

Centers for disease control and prevention. https://www.cdc.gov/drugoverdose/epidemic/index.html (2018, accessed 01 September 2020).

19.

National Center for Health Statistics. https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm (2021, accessed 01 November 2021).

20.

St Clair

Golub

et al. Characteristics associated with U.S. outpatient opioid analgesic prescribing and gabapentinoid co-prescribing. Am J Prev Med 2020; 58: e11–e19.

21.

Zou

. The adaptive lasso and its oracle properties. J Am Stat Assoc 2006; 101: 1418–1429.

22.

Fan

. Variable selection via nonconcave penalized likelihood and its oracle properties. J Am Stat Assoc 2001; 96: 1348–1360.

23.

Cluster TGWUHPC. Building a shared resource hpc center across university schools and institutes: a case study, 2020. https://arxiv.org/abs/2003.13629/. 2003.13629.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.18 MB

Standardization of continuous and categorical covariates in sparse penalized regressions

Abstract

Keywords

Get full access to this article

References

Supplementary Material