Variable selection is fundamental in high-dimensional statistical modelling, including non-and semiparametric regression. However, little work has been done for variable selection in a partially linear model (PLM). We propose and study a unified approach via double penalized least squares, retaining good features of both variable selection and model estimation in the framework of PLM. The proposed method is distinguished from others in that the penalty functions combine the l1 penalty coming from wavelet thresholding in the non-parametric component with the l1 penalty from the lasso in the parametric component. Simulations are used to investigate the performances of the proposed estimator in various settings, illustrating its effectiveness for simultaneous variable selection as well as estimation.
Akaike H (1973) Maximum likelihood identification of Gaussian autoregressive moving average models. Biometrika, 60, 255–65.
2.
Antoniadis A and Pham D-T (1998) Wavelet regression for random or irregular design. Computational Statistics and Data Analysis, 28, 333–69.
3.
Cai T and Brown L (1998) Wavelet shrinkage for nonequispaced samples. The Annals of Statistics, 26, 1783–99.
4.
Chang X and Qu L (2004) Wavelet estimation of partially linear models. Computational Statistics and Data Analysis, 47, 31–48.
5.
Cheng F (2005) On variance estimation in semiparametric regression models. Communications in Statistics—Theory and Methods, 34, 1737–42.
6.
Craven P and Wahba G (1979) Smoothing noisy data with spline functions. Numerische Mathematik, 31, 377–403.
7.
Donoho DL and Johnstone IM (1994) Ideal spatial adaptation via wavelet shrinkage. Biometrika, 81, 425–55.
8.
Donoho DL and Johnstone IM (1995) Adapting to unknown smoothness via wavelet shrinkage. Journal of the American Statistical Association, 90, 1200–24.
9.
Donoho DL, Johnstone IM, Kerkyacharian G and Picard D (1995) Wavelet shrinkage: asymptopia. Journal of the Royal Statistical Society, Series B, 57, 301–37.
10.
Efron B, Hastie T, Johnstone I and Tibshirani R (2004) Least angle regression. The Annals of Statistics, 32, 407–89.
11.
Fadili M (2005) Penalized partially linear models using sparse representations with an application to fMRI time series. IEEE Transactions on Signal Processing, 53, 3436–48.
12.
Fadili M and Bullmore E (2004) Penalized partially linear models using orthonormal wavelet bases with an application to fMRI time series. IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2, 1171–74.
13.
Fan J and Li R (2001) Variable selection via nonconvave penalized likelihood and its oracle properties. Journal ofthe American Statistical Association, 96, 1348–60.
14.
Fu WJ (1998) Penalized regression: the bridge versus the lasso. Journal ofComputational and Graphical Statistics, 7, 397–416.
15.
Hamilton SA and Truong YK (1997) Local linear estimation in partly linear models. Journal ofMultivariate Analysis, 60, 1–19.
16.
Härdle W, Liang H and Gao J (2000) Partially linear models. Heidelberg: Springer Verlag.
17.
Heckman N (1986) Spline smoothing in partly linear models. Journal ofthe Royal Statistical Society, Series B, 48, 244–48.
18.
Jansen M and Bultheel A (1999) Multiple wavelet threshold estimation by generalized cross validation for images with correlated noise. IEEE Transactions on Image Processing, 8, 947–53.
19.
Jansen M, Malfait M and Bultheel A (1997) Generalized cross validation for wavelet thresholding. Signal Processing, 56, 33–44.
20.
Knight K and Fu W (2000) Asymptotics for lasso-type estimators. The Annals of Statistics, 28, 1356–78.
21.
Kovac A and Silverman BW (2000) Extending the scope of wavelet regression methods by coeficient-dependent thresholding. Journal of the American Statistical Association, 95, 172–83.
22.
Li R and Liang H (2008) Variable selection in semiparametric regression modeling. The Annals of Statistics, 36, 261–86.
23.
Mallows C (1973) Some comments on Cp. Technometrics, 15, 661–75.
24.
Nason GP (1996) Wavelet shrinkage using cross validation. Journal ofthe Royal Statistical Society, Series B, 58, 463–79.
25.
Rice J (1986) Convergence rates for partially splined models. Statistics and Probability Letters, 4, 203–8.
26.
Schwarz G (1978) Estimating the dimension of a model. The Annals of Statistics, 6, 461–64.
27.
Simonoff JS and Tsai CL (1999) Semiparametric and additive model selection using an improved Akaike information criterion. Journal of Computational and Graphical Statistics, 8, 22–40.
28.
Speckman P (1988) Kernel smoothing in partial linear models. Journal of the Royal Statistical Society, Series B, 50, 413–36.
29.
Sweldens W (1998) The lifting scheme: a construction of second generation wavelets. SIAM Journal on Mathematical Analysis, 29, 511–46.
30.
Tibshirani R and Saunders M (2005) Sparsity and smoothness via the fused lasso. Journal of the Royal Statistical Society, Series B, 67, 91–108.
31.
Tibshirani RJ (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B, 58, 267–88.
32.
Vanraes E, Jansen M and Bultheel A (2002) Stabilizing wavelet transforms for non-equispaced data smoothing. Signal Processing, 82, 1979–90.
33.
Wang H, Li G and Tsai CL (2007) Regression coefficient and autoregressive order shrinkage and selection via lasso. Journal ofthe Royal Statistical Society, Series B, 69, (part 1), 63–78.
34.
Zhang HH and Lu W (2007) Adaptive-lasso for Cox’s proportional hazards model. Biometrika, 94, 691–703.
35.
Zhao P and Yu B (2006) On model selection consistency of Lasso. Journal ofMachine Learning Research, 7, 2541–63.
36.
Zou H (2006) The adaptive lasso and its oracle properties. Journal ofthe American Statistical Association, 101, 1418–29.
37.
Zou H, Hastie T and Tibshirani R (2007) On the “degrees of freedom” of the lasso. The Annals of Statistics, 35, 2173–92.
