In this paper, we focus on the modeling problem of estimating data with non-sparse structures, specifically focusing on biological data that exhibit a high degree of relevant features. Various fields, such as biology and finance, face the challenge of non-sparse estimation. We address the problems using the proposed method, called structured iterative division. Structured iterative division effectively divides data into non-sparse and sparse structures and eliminates numerous irrelevant variables, significantly reducing the error while maintaining computational efficiency. Numerical and theoretical results demonstrate the competitive advantage of the proposed method on a wide range of problems, and the proposed method exhibits excellent statistical performance in numerical comparisons with several existing methods. We apply the proposed algorithm to two biology problems, gene microarray datasets, and chimeric protein datasets, to the prognostic risk of distant metastasis in breast cancer and Alzheimer’s disease, respectively. Structured iterative division provides insights into gene identification and selection, and we also provide meaningful results in anticipating cancer risk and identifying key factors.
ZhuYBradicJ. Significance testing in non-sparse high-dimensional linear models. Electron J Stat2018; 12: 3312–3364.
2.
CheslerEJLuLShouSet al. Complex trait analysis of gene expression uncovers polygenic and pleiotropic networks that modulate nervous system function. Nat Genet2005; 37: 233–242.
3.
Van DamSVosaUvan der GraafA, et al. Gene co-expression analysis for functional classification and gene–disease predictions. Brief Bioinform2018; 19: 575–592.
4.
ConsortiumGArdlieKGDelucaDSet al. The genotype-tissue expression (gtex) pilot analysis: Multitissue gene regulation in humans. Science2015; 348: 648–660.
5.
ZhaoBZouF. On polygenic risk scores for complex traits prediction. Biometrics2022; 78: 499–511.
6.
GiannoneDLenzaMPrimiceriGE. Economic predictions with big data: The illusion of sparsity. Econometrica2021; 89: 2409–2437.
7.
FanJLiRZhangC-H, et al. Statistical foundations of data science. Florida: CRC press, 2020.
8.
ZhangLLinX. Some considerations of classification for high dimension low-sample size data. Stat Method Med Res2013; 22: 537–550.
9.
TsudaTImaizumiM. Benign overfitting of non-sparse high-dimensional linear regression with correlated noise, 2023. arXiv preprint arXiv:2304.04037.
10.
BoyleEALiYIPritchardJK. An expanded view of complex traits: From polygenic to omnigenic. Cell2017; 169: 1177–1186.
11.
BelloniAChernozhukovVHansenC. Inference on treatment effects after selection among high-dimensional controls. Rev Econ Stud2014; 81: 608–650.
12.
CaiTTGuoZ. Confidence intervals for high-dimensional linear regression: Minimax rates and adaptivity. Ann Stat2017; 45: 615–646.
13.
ZhengZLvJLinW. Nonsparse learning with latent variables. Oper Res2021; 69: 346–359.
HeYJaideeSGaoJ. Most powerful test against a sequence of high dimensional local alternatives. J Economet2023; 234: 151–177.
16.
ZhaoJZhouYLiuY. Estimation of linear functionals in high-dimensional linear models: From sparsity to nonsparsity. J Am Stat Assoc2023; 0: 1–13.
17.
ZhuYBradicJ. Two-sample testing in non-sparse high-dimensional linear models, 2016. arXiv preprint arXiv:1610.04580.
18.
BradicJFanJZhuY. Testability of high-dimensional linear models with nonsparse structures. Ann Stat2022; 50: 615.
19.
TibshiraniR. Regression shrinkage and selection via the lasso. J R Stat Soc: Ser B1996; 58: 267–288.
20.
ZouH. The adaptive lasso and its oracle properties. J Am Stat Assoc2006; 101: 1418–1429.
21.
YuanMLinY. Model selection and estimation in regression with grouped variables. J R Stat Soc Ser B: Stat Method2006; 68: 49–67.
22.
ZouHHastieT. Regularization and variable selection via the elastic net. J R Stat Soc Ser B: Stat Method2005; 67: 301–320.
23.
FanJLvJC. Sure independence screening for ultrahigh dimensional feature space. J R Stat Soc: Ser B2008; 70: 849–911.
24.
ChengSH-CHuangT-TChengY-H, et al. Validation of the 18-gene classifier as a prognostic biomarker of distant metastasis in breast cancer. PLoS ONE2017; 12: e0184372.
25.
ChengSHHorngC-FWestM, et al. Genomic prediction of locoregional recurrence after mastectomy in breast cancer. J Clin Oncol2006; 24: 4594–4602.
26.
HuangJMorehouseCStreicherKet al. Altered expression of insulin receptor isoforms in breast cancer. PLoS ONE2011; 6: e26177.
27.
KristensenLSJakobsenTHagerH, et al. The emerging roles of circrnas in cancer and oncology. Nat Rev Clin Oncol2022; 19: 188–206.
28.
XiaoWZhangGChenBet al. Characterization of frequently mutated cancer genes and tumor mutation burden in chinese breast cancer. Front Oncol2021; 11: 618767.
29.
YeLGuoLHeZet al. Upregulation of e2f8 promotes cell proliferation and tumorigenicity in breast cancer by modulating g1/s phase transition. Oncotarget2016; 7: 23757.
30.
NageleEHanMDeMarshallC, et al. Diagnosis of alzheimer’s disease based on disease-specific autoantibody profiles in human sera. PLoS ONE2011; 6: e23112.
31.
MungasD. In-office mental status testing: A practical guide. Geriatrics1991; 46: 00–00.
32.
McKhannGMKnopmanDSChertkowHet al. The diagnosis of dementia due to alzheimer’s disease: Recommendations from the national institute on aging-alzheimer’s association workgroups on diagnostic guidelines for alzheimer’s disease. Alzheimers Dement2011; 7: 263–269.
33.
BiswasAChakrabortySBaruahVJ. Estimation of the proportion of true null hypotheses under sparse dependence: Adaptive fdr controlling in microarray data. Stat Method Med Res2022; 31: 917–927.
34.
ChenXYangY. Local linear approximation with laplacian smoothing penalty and application in biology. Stat Method Med Res2023; 32: 1145–1158.
35.
YuSYangY. An iterative algorithm for high-dimensional linear models with both sparse and non-sparse structures, 2023. arXiv preprint arXiv:2311.05339.
36.
TsengP. Convergence of a block coordinate descent method for nondifferentiable minimization. J Optimiz Theory Appl2001; 109: 475–494.
37.
NegahbanSRavikumarPWainwrightMJ, et al. A unified framework for high-dimensional analysis of -estimators with decomposable regularizers. Stat Sci2012; 27: 1348–1356.
