Nowadays, Semi-Supervised Learning lies at the core of the Machine Learning field trying to effectively exploit unlabeled data as much as possible, together with a small amount of labeled data aiming to improve the predictive performance. Depending on the nature of the output class, Semi-Supervised Classification and Semi-Supervised Regression constitute the basic components of Semi-Supervised Learning. Various studies deal with the implementation of Semi-Supervised Classification techniques in many real world problems over the last two decades in contrast with Semi-Supervised Regression, which is deemed to be a more general and slightly touched case. This survey aims to provide a detailed review of Semi-Supervised Regression methods and implemented algorithms in recent years. Our in-depth study reveals the relatively few studies that deal with this specific problem. Moreover, we seek to classify these methods by proposing a schema and categorizing all the related methods that have been developed in recent years according to specific criteria.
AlpaydinE., Introduction to machine learning. MIT Press, 2010.
2.
ChangX., Shao-BoL. and Ding-XuanZ., Distributed i-supervised Learning with Kernel Ridge Regression, 18 (2017), 1–22.
3.
ZhuX. and GoldbergA.B., Introduction to i-Supervised Learning, Synth Lect Artif Intell Mach Learn3(1)(2009), 1–130.
4.
SeokK., I-supervised regression based on support vector machine, J Korean Data Inf Sci Soc25(2)(2014), 447–454.
5.
CortesC. and MohriM., On transductive regression, Proceedings of the 19th International Conference on Neural Information Processing Systems (2006). MIT Press. 305–312.
6.
YarowskyD., Unsupervised word sense disambiguation rivaling supervised methods, Proceedings of the 33rd annual meeting on Association for Computational Linguistics (1995), 189–196.
7.
BlumA. and MitchellT., Combining labeled and unlabeled data with co-training, in, Proceedings of the eleventh annual conference on Computational learning theory (1998), 92–100.
8.
ZhouZ.H. and LiM., Tri-training: Exploiting unlabeled data using three classifiers, IEEE Trans Knowl Data Eng17(11)(2005), 1529–1541.
9.
JoachimsT., Transductive Inference for Text Classification using Support Vector Machines, (1999), 200–209.
10.
TrigueroI., SáezJ.A., LuengoJ., GarcíaS. and HerreraF., On the characterization of noise filtersfor self-training i-supervised in nearest neighbor classification, Neurocomputing132 (2014), 30–41.
11.
MendenhallW., SincichT., A second course in statistics: Regression analysisPrentice Hall2003.
12.
RwebangiraM.R., LaffertyJ., Local Linear i-supervised Regression, 3(1521)2009Pittsburgh, PA.
13.
ZhouZ.-H. and LiM., I-supervised regression with co-training,. Morgan Kaufmann Publishers Inc., Proceedings of the 19th international joint conference on Artificial intelligence (2005), 908–913.
14.
TrigueroI., GarcíaS. and HerreraF., Self-labeled techniques for semi-supervised learning: Tomy, software and empirical study, Knowl Inf Syst42(2)(2015), 245–284.
KangP., KimD. and ChoS., I-supervised support vector regression based on self-training with labeluncertainty: An application to virtual metrology in iconductor manufacturing, Expert Syst Appl51 (2016), 85–106.
17.
Camps-VallsG., Munoz-MariJ., Gomez-ChovaL. and Calpe-MaravillaJ., I-Supervised Support Vector Biophysical Parameter Estimation in, IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium (2008), 1131–1134.
18.
HardleW. and MammenE., Comparing nonparametric versus parametric regression fits, Ann Stat21(4)(1993), 1926–1947.
19.
AltmanN.S., An introduction to kernel and nearest-neighbor nonparametric regression, Am Stat46(3)(1992), 175.
20.
SunS., A survey of multi-view machine learning, Neural Comput Appl23(7–8)(2013), 2031–2038.
21.
LevatícJ., CeciM., KocevD. and DžeroskiS., Knowledge Base d Systems Self-training for multi-targetregression with tree enbles, Knowledge-Based Syst123 (2017), 41–60.
22.
NajafabadiM.M., VillanustreF., KhoshgoftaarT.M., SeliyaN., WaldR. and MuharemagicE., Deep learning applications and challenges in big data analytics, J Big Data2(1)(2015), 1.
23.
NigamK. and GhaniR., Analyzing the effectiveness and applicability of co-training, in pp, Proceedings of the ninth international conference on Information and knowledge management (2000), 86–93.
24.
DengC. and GuoM.Z., Heidelberg, Tri-training and Data Editing Based Semi-supervised Clustering Algorithm Springer (2006), 641–651Berlin.
25.
ZhouY. and GoldmanS., Democratic co-learning, in, 16th IEEE International Conference on Tools with Artificial Intelligence (2004), 594–602.
26.
LiM. and ZhouZ.-H., Improve computer-aided diagnosis with machine learning techniques using undiagnosed samples, IEEE Trans Syst Man, Cybern -Part A Syst Humans37(6)(2007), 1088–1098.
27.
FaroukM., Abdel Hady and F. Schwenker, combining committee-based i-supervised learning and active learning, J Comput Sci Technol25(4)(2010), 681–698.
28.
BrefeldU., GärtnerT., SchefferT. and WrobelS., Efficient co-regularised least squares regression. in ’, 06 (2006), 137–144.
29.
WangX., FuL. and MaL., I-supervised support vector regression model for remote sensing water quality retrieving, Chinese Geogr Sci21(1)(2011), 57–64.
30.
ShengB. and ZhuH., The convergence rate of i-supervised regression with quadratic loss, Appl. Math Comput321 (2018), 11–24.
31.
BaoL., YuanX. and GeZ., Co-training partial least squares model for i-supervised soft sensor development, Chemom Intell Lab Syst147 (2015), 75–85.
32.
AbdiH., Partial least squares regression and projection on latent structure regression (PLS Regression), Wiley Interdiscip Rev Comput Stat2(1)(2010), 97–106.
33.
TangD., YuT.-H. and KimT.-K., Real-Time Articulated Hand Pose Estimation Using i-supervised Transductive Regression Forests, in, 2013 IEEE International Conference on Computer Vision (2013), 3224–3231.
34.
TangY., PanX., YuanY., YanP., LiL. and LiX., Local i-supervised regression for single-image super-resolution, in, 2011 IEEE 13th International Workshop on Multimedia Signal Processing (2011), 1–5.
35.
HadyM.F.A., SchwenkerF. and PalmG., Springer, Heidelberg, Semi-supervised learning for regression with co-training by committee (2009), 121–130Berlin.
36.
SunX., GongD. and ZhangW., Interactive genetic algorithms with large population and i-supervised learning, Appl Soft Comput J2 (2012), 3004–3013.
37.
LiY., ZhaH. and ZhouZ., Learning safe prediction for i-Supervised Regression, Proc 31th Conf Artif Intell (AAAI 2017) (2017), 2217–2223.
38.
GuoX. and UeharaK., Graph-based i-supervised regression and its extensions, IJACSA) Int J Adv Comput Sci Appl6(6): 2015.
39.
PozdnoukhovA. and BengioS., I-Supervised Kernel Methods for Regression Estimation, in pp. V-577-V-, IEEEInternational Conference on Acoustics Speed and Signal Processing Proceedings5 (2006), 580.
40.
HsuC., ChangC., LinC., A practical guide to support vector classification, 2010.
41.
XuS., AnX., QiaoX., ZhuL. and LiL., I-supervised Least-squares Support Vector Regression Machines, J Inf Comput Sci8(6)(2011), 885–892.
42.
VapnikV.N., , NY: Springer, The Nature of Statistical Learning Theory2000–New York.
43.
LinH., LinH., LinC.-J., A Study on Sigmoid Kernels for SVM and the Training of non-PSD Kernels by SMO-type Methods, 2003.
44.
AronszajnN., Theory of reproducing kernels, Trans Am Math Soc68(3)(1950), 337–337.
45.
Shawe-TaylorJ., CristianiniN., Kernel methods for pattern analysisCambridge University Press2004.
46.
AngeliniL., MarinazzoD., PellicoroM., StramagliaS., I-supervised learning by search of optimal target vector, 2007.
47.
ChapelleO., VapnikV., WestonJ., Transductive Inference for Estimating Values of Functions, 2000.
48.
SeokK.-H., Study on i-supervised local constant regression estimation, J Korean Data Inf Sci Soc23(3)(2012), 579–585.
49.
NadarayaE.A., On estimating regression, Theory Probab Its Appl9(1)(1964), 141–142.
50.
WatsonG.S., Smooth regression analysis, Indian J Stat Ser A26(4)(1964), 359–372.
51.
ShimingS., Xiang, F. Feiping Nie and C. Changshui Zhang, i-Supervised Classification via Local Spline Regression, IEEE Trans Pattern Anal Mach Intell32(11)(2010), 2039–2053.
52.
BennettK.P. and DemirizA., I-Supervised Support Vector Machines, (1999), 368–374.
53.
ZhuX. and GoldbergA.B., Kernel Regression with Order Preferences, in, AAAI (2007), 681–687.
54.
DemyanovV., PozdnoukhovA., KanevskiM., ChristieM.A., Geomodelling of a fluvial system with i-supervised support vector regression, 2008.
55.
BrouardC., SzafranskiM. and D’alché-bucF., I-supervised Penalized Output Kernel Regression for Link Prediction, (2011), 593–600.
56.
BelkinM., NiyogiP. and SindhwaniV., Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples, J Mach Learn Res7 (2006), 2399–2434.
57.
GuillauminM., VerbeekJ. and SchmidC., Multimodal i-supervised learning for image classification, in, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2010), 902–909.
58.
WassermanL. and LaffertyJ.D., Statistical Analysis of i-Supervised Regression, (2008), 801–808.
59.
WuM. and ScholkopfB., Transductive Classification via Local Learning Regularization, (2007), 628–635.
60.
VerbeekJ.J. and VlassisN., Gaussian fields for i-supervised regression and correspondence learning, Pattern Recognit39 (2006), 1864–1875.
61.
XieL., Carreira-PerpinanM.A. and NewsamS., “i-supervised regression with temporal image sequences, in, 2010 IEEE International Conference on Image Processing (2010), 2637–2640.
62.
XieL., NewsamS., WSM11 (2011), 29.
63.
DoquireG. and VerleysenM., A graph Laplacian based approach to i-supervised feature selection for regressionproblems, Neurocomputing121 (2013), 5–13.
64.
ZhaoM., ChowT.W.S., WuZ., ZhangZ. and LiB., Learning from normalized local and global discriminativeinformation for i-supervised regression and dimensionality reduction, Inf Sci (Ny)324 (2015), 286–309.
65.
ZhaoM., LiB., WuZ. and ZhanC., Image classification via least square i-supervised discriminant analysis with flexible kernel regression for out-of-sample extension, Neurocomputing153 (2015), 96–107.
66.
KimK.I., SteinkeF. and HeinM., I-supervised Regression using Hessian energy with an application to i-supervised dimensionality reduction, (2009), 979–987.
67.
LinB., ZhangC. and HeX., I-supervised Regression via Parallel Field Regularization, (2011), 433–441.
CaiD., HeX., HanJ., I-Supervised Regression using Spectral Techniques, 2006.
70.
CaiD., HeX. and HanJ., I-supervised Discriminant Analysis, in, 2007 IEEE 11th International Conference on Computer Vision (2007), 1–7.
71.
GoldbergA., ZhuX., SinghA., XuZ. and NowakR., Multi-Manifold i-Supervised Learning, J Mach Learn Res Work Conf Proc (2009).
72.
SeahC.-W., TsangI.W. and OngY.-S., Transductive ordinal regression, IEEE Trans Neural Networks Learn Syst23(7)(2012), 1074–1086.
73.
RasmussenC.E., WilliamsC.K.I., Gaussian processes for machine learningMIT Press, 2006.
74.
PicharaK. and SotoA., Local feature selection using Gaussian process regression, Intell Data Anal18(3)(2014), 319–336.
75.
ZhangY. and YeungD.-Y., I-Supervised Multi-Task Regression, (2009), 617–631.
76.
NgM.K., ChanE.Y., SoM.M.C. and ChingW.-K., A i-supervised regression model for mixed numerical and categorical variables, Pattern Recognit40 (2007), 1745–1752.
77.
ChuW. and GhahramaniZ., Gaussian processes for ordinal regression, J Mach Learn Res6 (2007), 1019–1041.
78.
SrijithP.K., ShevadeS. and SundararajanS., I-supervised Gaussian Process Ordinal Regression, (2013), 144–159.
79.
UtoK., KosugiY. and SaitoG., I-supervised hyperspectral subspace learning based on a generalized eigenvalue problem for regression and dimensionality reduction, IEEE J Sel Top Appl Earth Obs Remote Sens7(6)(2014), 2583–2599.
