We present a novel kernel-free regressor, called quadratic hyper-surface kernel-free least squares support vector regression (QLSSVR), for some regression problems. The task of this approach is to find a quadratic function as the regression function, which is obtained by solving a quadratic programming problem with the equality constraints. Basically, the new model just needs to solve a system of linear equations to achieve the optimal solution instead of solving a quadratic programming problem. Therefore, compared with the standard support vector regression, our approach is much efficient due to kernel-free and solving a set of linear equations. Numerical results illustrate that our approach has better performance than other existing regression approaches in terms of regression criterion and CPU time.
AbeS., Fuzzy support vector machines for multilabel classification, Pattern Recognition48(6) (2015), 2110–2117.
2.
AKS.J. et al., Least squares support vector machines, World Scientific, 2002.
3.
BaiY.HanX.ChenT. and YuH., Quadratic kernel-free least squares support vector machine for target diseases classification, Journal of Combinatorial Optimization30(4) (2015), 850–870.
4.
BalasundaramS.GuptaD. and Kapil, Lagrangian support vector regression via unconstrained convex minimization, Neural Netw.51 (Mar. 2014), 67–79.
5.
BoserB.E.GuyonI.M. and VapnikV.N., A training algorithm for optimal margin classifiers, in: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, ACM, 1992, pp. 144–152.
6.
BurgesC.J., A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery2(2) (1998), 121–167.
7.
CaiY.WangH.YeX. and FanQ., A multiple-kernel lssvr method for separable nonlinear system identification, Journal of Control Theory and Applications11(4) (2013), 651–655.
8.
CherkasskyV. and MulierF., Learning from data: concepts, theory, and methods john wiley and sons, New York, 1998.
9.
ClevelandW.GrosseE. and ShyuW., Local regression models. statistical models in s (chambers, jm and hastie, tj, eds), 309–376, Wadsworth & Brooks, Pacific Grove, 1991.
10.
CortesC. and VapnikV., Support-vector networks, Machine Learning20(3) (1995), 273–297.
11.
CoverT.M., Geometrical and statistical properties of systems of linear inequalities with applications in pattern recognition, IEEE Transactions on Electronic Computers (3) (1965), 326–334.
12.
CrammerK. and SingerY., On the algorithmic implementation of multiclass kernel-based vector machines, Journal of Machine Learning Research2(Dec) (2001), 265–292.
13.
DagherI., Quadratic kernel-free non-linear support vector machine, Journal of Global Optimization41(1) (2008), 15–30.
14.
DruckerH.BurgesC.J.KaufmanL.SmolaA.J. and VapnikV., Support vector regression machines, in: Advances in Neural Information Processing Systems, 1997, pp. 155–161.
15.
FanJ.HuangT. et al., Profile likelihood inferences on semiparametric varying-coefficient partially linear models, Bernoulli11(6) (2005), 1031–1057.
16.
FletcherR., Practical methods of optimization, John Wiley & Sons, 2013.
17.
GlielmoA.SollichP. and De VitaA., Accurate interatomic force fields via machine learning with covariant kernels, Physical Review B95(21) (2017), 214302.
18.
HofmannT.SchölkopfB. and SmolaA.J., Kernel methods in machine learning, in: The Annals of Statistics, 2008, pp. 1171–1220.
19.
IzoninI.TrostianchynA.DuriaginaZ.TkachenkoR.TeplaT. and LotoshynskaN., The combined use of the wiener polynomial and svm for material classification task in medical implants production, Int. J. Intell. Syst. Appl. (IJISA)10(9) (2018), 40–47.
20.
KaralO., Maximum likelihood optimal and robust support vector regression with lncosh loss function, Neural Networks94 (2017), 1–12.
21.
KhemchandaniR.GoyalK. and ChandraS., Twsvr: regression via twin support vector machine, Neural Networks74 (2016), 14–21.
22.
LeT.NguyenT.D.NguyenV. and PhungD., Approximation vector machines for large-scale online learning, The Journal of Machine Learning Research18(1) (2017), 3962–4016.
23.
HuberM.B.LancianeseS.L.NagarajanM.B.IkpotI.Z.LernerA.L. and WismullerA., Prediction of biomechanical properties of trabecular bone in mr images with geometric features and support vector regression, IEEE Transactions on Biomedical Engineering58(6) (June 2011), 1820–1826.
24.
MahmoodianH. and EbrahimianL., Using support vector regression in gene selection and fuzzy rule generation for relapse time prediction of breast cancer, Biocybernetics and Biomedical Engineering36(3) (2016), 466–472.
25.
MarquardtD.W., An algorithm for least-squares estimation of nonlinear parameters, Journal of the Society for Industrial and Applied Mathematics11(2) (1963), 431–441.
26.
NaumovichV.V.VapnikV. et al., Statistical learning theory, 1998.
27.
NelderJ.A. and WedderburnR.W., Generalized linear models, Journal of the Royal Statistical Society: Series A (General)135(3) (1972), 370–384.
28.
PengX., Tsvr: an efficient twin support vector machine for regression, Neural Networks23(3) (2010), 365–372.
29.
PingM.M. and XiaY.Z., -kernel-free soft quadratic surface support vector regression, Submitted.
30.
HuQ.ZhangS.YuM. and XieZ., Short-term wind speed or power forecasting with heteroscedastic support vector regression, IEEE Transactions on Sustainable Energy7 (11 2015), 1–9.
31.
HuQ.ZhangS.XieZ.MiJ. and WanJ., Noise model based ν-support vector regression with its application to short-term wind speed forecasting, Neural Networks57 (Sept. 2014), 1–11.
32.
SchabackR. and WendlandH., Kernel techniques: from machine learning to meshless methods, Acta Numerica15 (2006), 543–639.
33.
SteinwartI., Sparseness of support vector machines, Journal of Machine Learning Research4(Nov) (2003), 1071–1105.
34.
SuykensJ.A. and VandewalleJ., Least squares support vector machine classifiers, Neural Processing Letters9(3) (1999), 293–300.
35.
VapnikV.GolowichS.E. and SmolaA.J., Support vector method for function approximation, regression estimation and signal processing, in: Advances in Neural Information Processing Systems, 1997, pp. 281–287.
