The concept of a large margin is central to support vector machines and it has recently been adapted and applied for nearest neighbour classification. In this paper, a modification of this method is proposed in order to be used for regression problems. This model also allows the use of a set of prototypes with different distance metrics, which can increase the flexibility of the method especially for problems with large number of instances. The learning of the distance metrics is performed by two optimization methods, namely an evolutionary algorithm and an approximate differential approach. A real world problem, i.e. the prediction of the corrosion resistance of some alloys containing titanium and molybdenum is considered as a case study. It is shown that the suggested method provides very good results compared to other well-known regression algorithms.
AhaD.W. and GoldstoneR.L., Concept learning and flexible weighting, Proceedings of the 14th Annual Conference of the Cognitive Science Society, 1992, pp. 534–539.
2.
AssiK.C., LabelleH. and CherietF., Modified large margin nearest neighbor metric learning for regression, IEEE Signal Processing Letters21(3) (2014), 292–296. doi: 10.1109/LSP.2014.2301037
3.
AtkesonC.G., MooreA.W. and SchaalS., Locally weighted learning, Artificial Intelligence Review 11(1-5), Kluwer Academic PublishersNorwell, MA, USA, 1997, pp. 11–73.
4.
ChelariuR., SudituG.D., MareciD., BolatG., CimpoesuN., LeonF. and CurteanuS., Prediction of corrosion resistance of some dental metallic materials applying artificial neural networks and regression based models, JOM The Journal of the Minerals, Metals & Materials Society67(4) (2015), 767–774.
5.
ChopraS., HadsellR. and LeCunY., Learning a similarity metric discriminatively, with application to face verification, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR-05, San Diego, CA, USA, 2005, pp. 349–356.
6.
CoverT.M. and HartP.E., Nearest neighbor pattern classification, IEEE Transactions in Information Theory IT-13 (1967), 21–27.
7.
GoldbergerJ., RoweisS., HintonG. and SalakhutdinovR., Neighbourhood components analysis, Advances in Neural Information Processing Systems 17, Cambridge, MA, USA, MIT Press, 2005, pp. 513–520.
8.
HallM., FrankE., HolmesG., PfahringerB., ReutemannP. and WittenI.H., The WEKA data mining software: An update, ACM SIGKDD Explorations11(1) (2009), 10–18.
JeffreysH. and JeffreysB.S., Central Differences Formula, in Methods of Mathematical Physics, 3rd edition, Cambridge University Press, 1988, pp. 284–286.
11.
KumarM.P., TorrP.H.S. and ZissermanA., An Invariant Large Margin Nearest Neighbour Classifier, Proceedings of the IEEE 11th International Conference on Computer Vision, ICCV 2007, Rio de Janeiro, Brazil, 2007. doi: 10.1109/iccv.2007.4409041
12.
LeonF. and CurteanuS., Evolutionary Algorithm for Large Margin Nearest Neighbour Regression, Proceedings of the 7th International Conference on Computational Collective Intelligence Technologies and Applications, ICCCI 2015, Madrid, Spain, Part I, Lecture Notes in Artificial Intelligence, LNAI9329, Springer International Publishing Switzerland, 2015, pp. 286–296. doi: 10.1007/978-3-319-24069-5_29
13.
LeonF., PiuleacC.G., CurteanuS. and PouliosI., Instance-based regression with missing data applied to a photocatalytic oxidation process, Central European Journal of Chemistry10(4) (2012), 1149–1156. doi: 10.2478/s11532-012-0038-x
14.
MareciD., SutimanD., ChelariuR., LeonF. and CurteanuS., Evaluation of the corrosion resistance of new TiZr binary alloys by experiment and simulation based on regression model with incomplete data, Corrosion Science73 (2013), 106–122. Elsevier, doi: 10.1016/j.corsci.2013.03.030
15.
MooreR.C. and DeNeroJ., L1 and L2 regularization for multiclass hinge loss models, Proceedings of the Symposium on Machine Learning in Speech and Language Processing, 2011.
16.
NavotA., ShpigelmanL., TishbyN. and VaadiaE., Nearest neighbor based feature selection for regression and its application to neural activity, Advances in Neural Information Processing Systems19 (2005).
17.
Shalev-ShwartzS., SingerY. and NgA.Y., Online and batch learning of pseudo-metrics, Proceedings of the 21st International Conference on Machine Learning, ICML-04, Banff, Canada, 2004, pp. 94–101.
18.
ShentalN., HertzT., WeinshallD. and PavelM., Adjustment learning and relevant component analysis, Proceedings of the 7th European Conference on Computer Vision, ECCV-02, vol. 4, London, UK, Springer-Verlag, 2002, pp. 776–792
.
19.
WeinbergerK.Q., BlitzerJ. and SaulL.K., Distance metric learning for large margin nearest neighbor classification, Advances in Neural Information Processing Systems 18, Cambridge, MA, USA, MIT Press, 2006, pp. 1473–1480.
20.
WeinbergerK.Q. and SaulL.K., Fast solvers and efficient implementations for distance metric learning, Proceedings of the 25th International Conference on Machine Learning, Helsinki, Finland, 2008, pp. 1160–1167.
21.
WeinbergerK.Q. and SaulL.K., Distance metric learning for large margin nearest neighbor classification, Journal of Machine Learning Research10 (2009), 207–244.
