Researchers are increasingly interested in regression models for functional data. This article discusses a comprehensive framework for additive (mixed) models for functional responses and/or functional covariates based on the guiding principle of reframing functional regression in terms of corresponding models for scalar data, allowing the adaptation of a large body of existing methods for these novel tasks. The framework encompasses many existing as well as new models. It includes regression for ‘generalized’ functional data, mean regression, quantile regression as well as generalized additive models for location, shape and scale (GAMLSS) for functional data. It admits many flexible linear, smooth or interaction terms of scalar and functional covariates as well as (functional) random effects and allows flexible choices of bases—particularly splines and functional principal components—and corresponding penalties for each term. It covers functional data observed on common (dense) or curve-specific (sparse) grids. Penalized-likelihood-based and gradient-boosting-based inference for these models are implemented in R packages refund and FDboost, respectively. We also discuss identifiability and computational complexity for the functional regression models covered. A running example on a longitudinal multiple sclerosis imaging study serves to illustrate the flexibility and utility of the proposed model class. Reproducible code for this case study is made available online.
BrockhausSFuestAMayrAGrevenS (2015a) Functional regression models for location, scale and shape with application to stock returns. In FriedlHWagnerH , eds. Proceedings of the 30th International Workshop on Statistical Modelling, vol. 1, Linz, July 6–10, 2015, pages 117–22.
2.
BrockhausSFuestAMayrAGrevenS (2016a) Signal regression models for loca- tion, scale and shape with an application to stock returns. arXiv preprint arXiv:1605. 04281.
3.
BrockhausSMelcherMLeischFGrevenS (2016b) Boosting flexible functional regre- ssion models with a high number of functional historical effects. Statistics and Computing. Available at http://link.springer.com/article/10.1007/s11222-016-9662-1. (accessed on 17 November 2016).
4.
BrockhausSRügamerD (2016) FDboost: Boosting functional regression models. R package version 0.2-0. Available at http://CRAN.R-project.org/package=FDboost. (accessed on 17 November 2016).
5.
BrockhausSScheiplFHothornTGrevenS (2015b) The functional linear array model. Statistical Modelling, 15, 279–300.
6.
BühlmannPHothornT (2007) Boosting algo- rithms: Regularization, prediction and model fitting. Statistical Science, 22, 477–505.
7.
CardotHFerratyFSardaP (1999) Functional linear model. Statistics & Probability Letters, 45, 11–22.
8.
CederbaumJPouplierMHoolePGrevenS (2016) Functional linear mixed models for irregularly or sparsely sampled data. Statistical Modelling, 16, 67–88.
9.
CurrieIDDurbanMEilersPH (2006) Gene- ralized linear array models with applications to multidimensional smoothing. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68, 259–80.
10.
De BoorC (1978) A Practical Guide to Splines. New York: Springer.
11.
DiC-ZCrainiceanuCMCaffoBSPunjabiNM (2009) Multilevel functional principal component analysis. The Annals of Applied Statistics, 3, 458–88.
12.
DrydenIL (2014) Shape and object data analysis. Biometrical Journal, 56, 758–60.
13.
EilersPMarxB (1996) Flexible smoothing with B-splines and penalties. Statistical Sciences, 11, 89–121.
14.
EilersPHMarxBD (2003) Multivariate calibration with temperature interaction using two-dimensional penalized signal regression. Chemometrics and intelligent laboratory systems, 66, 159–74.
15.
EilersPHCMarxBD (2002) Generalized linear additive smooth structures. Journal of Computational and Graphical Statistics, 11, 758–83.
Febrero-BandeMOviedo de la FuenteM (2012) Statistical computing in functional data analysis: The R package fda.usc. Journal of Statistical Software, 51, 1–28. Available at http://www.jstatsoft.org/v51/i04/ (accessed on 17 November 2016).
18.
FenskeNKneibTHothornT (2011) Identifying risk factors for severe childhood malnutrition by boosting additive quantile regression. Journal of the American Statistical Association, 106, 494–510.
19.
FerratyFVieuP (2006) Nonparametric Functional Data Analysis: Theory and Practice. New York: Springer.
20.
FuchsKScheiplFGrevenS (2015) Penalized scalar-on-functions regression with interaction term. Computational Statistics & Data Analysis, 81, 38–51.
21.
GasparriniAArmstrongBKenwardMG (2010) Distributed lag non-linear models. Statistics in Medicine, 29, 2224–34.
22.
GertheissJGoldsmithJCrainiceanuCGrevenS (2013a) Longitudinal scalar-on- functions regression with application to tractography data. Biostatistics, 14, 447–61.
23.
GertheissJMaityAStaicuA-M (2013b) Variable selection in generalized functional linear models. Stat, 2, 86–101.
GoldsmithJBobbJCrainiceanuCMCaffoBReichD (2012) Penalized functional regression. Journal of Computational and Graphical Statistics, 20, 830–51.
26.
GoldsmithJGrevenSCrainiceanuCM (2013) Corrected confidence bands for functional data using principal components. Biometrics, 69, 41–51.
27.
GoldsmithJHuangLCrainiceanuCM (2014) Smooth scalar-on-image regression via spatial Bayesian variable selection. Journal of Computational and Graphical Statistics, 23, 46–64.
28.
GoldsmithJWandMPCrainiceanuCM (2011) Functional regression via variational Bayes. Electronic Journal of Statistics, 5, 572–602.
29.
GoldsmithJZipunnikovVSchrackJ (2015) Generalized multilevel function-on-scalar regression and principal component analysis. Biometrics, 71, 344–53.
30.
GrevenSCrainiceanuCMCaffoBSReichD (2010) Longitudinal functional principal component analysis. Electronic Journal of Statistics, 4, 1022–54.
31.
GrevenSCrainiceanuCMKüchenhoffHPetersA (2008) Restricted likelihood ratio testing for zero variance components in linear mixed models. Journal of Computational and Graphical Statistics, 17, 870–91.
32.
HadjipantelisPZAstonJAMüllerH-GEvansJP (2015) Unifying amplitude and phase analysis: A compositional data approach to functional multivariate mixed- effects modeling of mandarin Chinese. Journal of the American Statistical Association, 110, 545–59.
33.
HallPHookerG (2016) Truncated linear models for functional data. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 78, 637–53. Available at http://onlinelibrary.wiley.com/doi/10.1111/ rssb.12125/abstract
34.
HastieTMallowsR (1993) Discussion of ‘A statistical view of some chemometrics reg- ression tools,’ by I. E. Frank and J. H. Friedman. Technometrics, 35, 140–43.
35.
HastieTTibshiraniR (1993) Varying-coefficient models. Journal of the Royal Statistical Society. Series B, 55, 757–96.
36.
HastieTJTibshiraniRJFriedmanJH (2011) The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York: Springer.
37.
HerrickR (2015) WFMM. The University of Texas M.D. Anderson Cancer Center, version 3.0 edition. Available at https://biostatistics.mdanderson.org/SoftwareDownload/SingleSoftware.aspx?Software_Id=70 (accessed on 16 December 2016)
38.
HofnerBHothornTKneibTSchmidM (2012) A framework for unbiased model selection based on boosting. Journal of Computational and Graphical Statistics, 20, 956–71.
39.
HofnerBMayrAFenskeNSchmidM (2016) gamboostLSS: Boosting Methods for GAMLSS Models.R package version 1.3-0. Available at http://CRAN.R-project.org/package=gamboostLSS (accessed on 29 October 2016).
40.
HorváthLKokoszkaP (2012) Inference for Functional Data with Applications. New York: Springer Science & Business Media.
41.
HothornTBuehlmannPKneibTSchmidMHofnerB (2016) mboost: Model-Based Boosting. R package version 2.6-0. Available at http://CRAN.R-project.org/package=mboost (accessed on 22 November 2016).
42.
HothornTKneibTBühlmannP (2014) Conditional transformation models. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 76, 3–27.
43.
HuangLScheiplFGoldsmithJGellarJHarezlakJMcLeanMWSwihartBXiaoLCrainiceanuCReissP (2016) refund: Regression with Functional Data. R package version 0.1-15. Available at https://CRAN.R-project.org/package=refund. (accessed on 22 November 2016).
JamesGMHastieTJSugarCA (2000) Principal component models for sparse functional data. Biometrika, 87, 587–602.
46.
JamesGMSilvermanBW (2005) Functional adaptive model estimation. Journal of the American Statistical Association, 100, 565–76.
47.
KarhunenK (1947) Über Lineare Methoden in der Wahrscheinlichkeitsrechnung. Annales Academiae Scientiarum Fennicae, 37, 1–79.
48.
KneibTHothornTTutzG (2009) Variable selection and model choice in geoadditive regression models. Biometrics, 65, 626–34.
49.
KoenkerR (2005) Quantile Regression. Cambridge: Cambridge University Press.
50.
LangSBrezgerA (2004) Bayesian P-splines. Journal of Computational and Graphical Statistics, 13, 183–212.
51.
LoèveM (1945) Fonctions aléatoires de second ordre. Comptes Rendus Académie des Sciences, 220, 469.
52.
MalfaitNRamsayJO (2003) The historical functional linear model. Canadian Journal of Statistics, 31, 115–28.
53.
MarraGWoodSN (2012) Coverage proper- ties of confidence intervals for generalized additive model components. Scandinavian Journal of Statistics, 39, 53–74.
54.
MarronJSAlonsoAM (2014) Overview of object oriented data analysis. Biometrical Journal, 56, 732–53.
55.
MarronJSRamsayJOSangalliLMSrivastavaA (2014) Statistics of time warpings and phase variations. Electronic Journal of Statistics, 8, 1697–702.
56.
MarxBDEilersPH (1999) Generalized linear regression on sampled signals and curves: A P-spline approach. Technometrics, 41, 1–13.
57.
MarxBDEilersPH (2005) Multidimensional penalized signal regression. Technometrics, 47, 13–22.
58.
MayrAFenskeNHofnerBKneibTSchmidM (2012) Generalized additive models for location, scale and shape for high dimensional data—A flexible approach based on boosting. Journal of the Royal Statistical Society: Series C (Applied Statistics), 61, 403–27.
59.
McLeanMWHookerGRuppertD (2015) Restricted likelihood ratio tests for linearity in scalar-on-function regression. Statistics and Computing, 25, 997–1008.
60.
McLeanMWHookerGStaicuA-MScheiplFRuppertD (2014) Functional generalized additive models. Journal of Computational and Graphical Statistics, 23, 249–69.
61.
MeinshausenNBühlmannP (2010) Stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72, 417–73.
62.
MercerJ (1909) Functions of positive and negative type, and their connection with the theory of integral equations. Philosophical Transactions of the Royal Society of London. Series A, 209, 415–46.
63.
MeyerMJCoullBAVersaceFCinciripiniPMorrisJS (2015) Bayesian function-on- function regression for multilevel functional data. Biometrics, 71, 563–74.
64.
MorrisJS (2015) Functional regression. Annual Review of Statistics and its Applications, 2, 321–59.
65.
MorrisJSArroyoCCoullBARyanLMHerrickRGortmakerSL (2006) Using wavelet-based functional mixed models to characterize population heterogeneity in accelerometer profiles: A case study. Journal of the American Statistical Association, 101, 1352–64.
66.
MorrisJSBaladandayuthapaniVHerrickRCSannaPGutsteinH (2011) Automated analysis of quantitative image data using isomorphic functional mixed models, with application to proteomics data. The Annals of Applied Statistics, 5, 894–923.
67.
MorrisJSCarrollRJ (2006) Wavelet-based functional mixed models. Journal of the Royal Statistical Society, Series B, 68, 179–99.
68.
MüllerH-GStadtmüllerU (2005) Genera- lized functional linear models. Annals of Statistics, 33, 774–805.
69.
ObermeierVScheiplFHeumannCWassermannJKüchenhoffH (2015) Flexible dis- tributed lags for modelling earthquake data. Journal of the Royal Statistical Society: Series C (Applied Statistics), 64, 395–412.
70.
O'SullivanF (1986) A statistical perspective on ill-posed inverse problems. Statistical Science, 1, 502–18.
71.
PengJPaulD (2012) A geometric approach to maximum likelihood estimation of the functional principal components from sparse longitudinal data. Journal of Compu- tational and Graphical Statistics, 18, 995–1015.
72.
PlummerM (2016) rjags: Bayesian Graphical Models Using MCMC. R package version 4-6. Available at https://CRAN.R-project.org/package=rjags. (accessed on 23 November 2016).
73.
RamsayJODalzellC (1991) Some tools for functional data analysis. Journal of the Royal Statistical Society. Series B (Methodological), 33, 539–72.
74.
RamsayJGravesSHookerG (2009) Functional data analysis with R and MATLAB. New York: Springer.
75.
RamsayJOSilvermanBW (2005) Functional Data Analysis. New York: Springer.
76.
RamsayJOWickhamHGravesSHookerG (2014) fda: Functional Data Analysis. R package version 2.4-4. Available at http://CRAN.R-project.org/package=fda. (accessed on 23 November 2016).
77.
ReimherrMNicolaeD (2016) Estimating variance components in functional linear models with applications to genetic heritability. Journal of the American Statis- tical Association, 111, 407–22.
78.
ReissPTOgdenRT (2007) Functional principal component regression and functional partial least squares. Journal of the American Statistical Association, 102, 984–96.
79.
ReissPTGoldsmithJShangHLOgdenRT (2016) Methods for scalar-on-function regression. International Statistical Review. Available at http://onlinelibrary.wiley.com/doi/10.1111/insr.12163/full
80.
ReissPTHuangLMennesM (2010) Fast function-on-scalar regression with penalized basis expansions. The International Journal of Biostatistics, 6, 1557–4679.
81.
ReissPTMillerDLWuP-SHuaW-Y (2015) Penalized nonparametric scalar-on-function regression via principal coordinates. Technical Report. Available at http://works.bepress.com/phil_reiss/42/ (accessed on 23 November 2016).
82.
ReissPTOgdenRT (2007) Functional prin- cipal component regression and functional partial least squares. Journal of the American Statistical Association, 102, 984–96.
83.
RigbyRAStasinopoulosDM (2005) Generalized additive models for location, scale and shape. Journal of the Royal Statistical Society: Series C (Applied Statistics), 54, 507–54.
84.
RuppertDWandMPCarrollRJ (2003) Semiparametric Regression. Cambridge: Cambridge University Press.
ScheiplFGrevenS (2016) Identifiability in penalized function-on-function regression models. Electronic Journal of Statistics, 10, 495–526.
87.
ScheiplFGrevenSKüchenhoffH (2008) Size and power of tests for a zero random effect variance or polynomial regression in additive and linear mixed models. Computational Statistics & Data Analysis, 52, 3283–99.
88.
ScheiplFStaicuA-MGrevenS (2015) Functional additive mixed models. Journal of Computational and Graphical Statistics, 24, 477–501.
89.
SchmidMHothornT (2008) Boosting additive models using component-wise P-splines. Computational Statistics & Data Analysis, 53, 298–311.
90.
ShahRDSamworthRJ (2013) Variable selection with error control: another look at stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 75, 55–80.
91.
ShiJQChengY (2014) GPFDA: Apply Gaussian Process in Functional data analysis. R package version 2.2. Available at https://CRAN.R-project.org/package=GPFDA (accessed on 23 November 2016).
92.
ShiJQChoiT (2011) Gaussian Process Regression Analysis for Functional Data. Boca Raton, FL: CRC Press.
93.
ShiJQWangBMurray-SmithRTitteringtonM (2007) Gaussian process functional regression modeling for batch data. Biometrics, 63, 714–23.
94.
ShouHZipunnikovVCrainiceanuCMGrevenS (2015) Structured functional principal component analysis. Biometrics, 71, 247–57.
95.
StaicuA-MLiYCrainiceanuCMRuppertD (2014) Likelihood ratio tests for dependent data with applications to longitudinal and functional data analysis. Scandinavian Journal of Statistics, 41, 932–49.
96.
StaniswalisJLeeJ (1998) Nonparametric regression analysis of longitudinal data. Journal of the American Statistical Association, 93, 1403–04.
97.
SwihartBJGoldsmithJCrainiceanuCM (2014) Restricted likelihood ratio tests for functional effects in the functional linear model. Technometrics, 56, 483–93.
98.
UssetJStaicuA-MMaityA (2016) Interaction models for functional regression. Computational Statistics & Data Analysis, 94, 317–30.
99.
Van der LindeA (2009) Bayesian functional principal components analysis for binary and count data. Advances in Statistical Analysis, 93, 307–33.
100.
WandMOrmerodJ (2008) On semiparametric regression with O'Sullivan penalized splines. Australian & New Zealand Journal of Statistics, 50, 179–98.
101.
WangBShiJQ (2014) Generalized Gaussian process regression model for non-Gaussian functional data. Journal of the American Statistical Association, 109, 1123–33.
102.
WangHMarronJ (2007) Object-oriented data analysis: Sets of trees. The Annals of Statistics, 35, 1849–73.
103.
WangJ-LChiouJ-MMuellerH-G (2016) Review of functional data analysis. Annual Review of Statistics and Its Application, 3, 257–95.
104.
WoodSN (2006a) Generalized Additive Models: An Introduction with R. Boca Raton, FL: CRC Press.
WoodSN (2011) Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models. Journal of the Royal Statistical Society: Series B, 73, 3–36.
107.
WoodSN (2013) A simple test for random effects in regression models. Biometrika, 100, 1005–10.
108.
WoodSN (2016a) Just another Gibbs additive modeller: Interfacing JAGS and mgcv. arXiv preprint arXiv:1602.02539. Available at https://arxiv.org/abs/1602.02539 (accessed on 23 November 2016).
109.
WoodSN (2016b) mgcv: Mixed GAM Computation Vehicle with GCV/AIC/REML Smoothness Estimation. R package version 1.8-12. Available at http://CRAN.R-project.org/package=mgcv (accessed on 23 November 2016).
110.
WoodSNLiZ, ShaddickGAugustinNH (2016a) Generalized additive models for gigadata: Modelling the UK black smoke network daily data. Journal of the Ameri- can Statistical Association. Available at http://amstat.tandfonline.com/doi/abs/10.1080/01621459.2016.1195744
111.
WoodSNPyaNSäfkenB (2016b) Smoothing parameter and model selection for general smooth models. Journal of the American Statistical Association. Available at http://arxiv.org/abs/1511.03864 (accessed on 23 November 2016).
112.
YaoFMüllerHWangJ (2005a) Functional data analysis for sparse longitudinal data. Journal of the American Statistical Association, 100, 577–90.
113.
YaoFMüllerHWangJ (2005b) Functional linear regression analysis for longitudinal data. The Annals of Statistics, 33, 2873–903.
114.
ZipunnikovVCaffoBYousemDMDavatzikosCSchwartzBSCrainiceanuC (2011) Multilevel functional principal component analysis for high-dimensional data. Journal of Computational and Graphical Statistics, 20, 852–73.
115.
ZipunnikovVGrevenSShouHCaffoBReichDSCrainiceanuC (2014) Longitudinal high-dimensional principal components analysis with application to diffusion tensor imaging of multiple sclerosis. The Annals of Applied Statistics, 8, 2175–202.
