Hybrid classification and regression models via particle swarm optimization auto associative neural network based nonlinear PCA

Abstract

For solving classification and regression problems, we propose a hybrid system consisting of two phases which work in tandem. In the first phase, particle swarm optimization is employed to train a 3-layered auto associative neural network (henceforth called PSOAANN). In this phase, dimensionality reduction takes place in hidden layer, where the hidden nodes should be less than the input nodes. The outputs from the hidden nodes are then treated as nonlinear principal components (NLPC). They are fed to the second phase where several classifiers and regression methods are invoked. The second phase includes techniques viz., threshold accepting logistic regression (TALR), probabilistic neural network (PNN), group method of data handling (GMDH), support vector machine (SVM) and genetic programming (GP) for classification problems. For regression problems, general regression neural network (GRNN) is used in place of PNN. In addition, support vector machine (SVM), Genetic Programming (GP), GMDH are also employed, as they are versatile. The efficiency of the hybrid is analyzed on five banking datasets namely Spanish banks, Turkish banks, US banks and UK banks and UK credit dataset and five regression datasets viz., Bodyfat, Forestfires, AutoMPG, Boston Housing and Pollution. All the datasets are analyzed using 10 fold cross validation (10 FCV). It turns out that the proposed hybrid yielded higher accuracies across classification and regression problems.