The complexity of certain problems causes that classical methods for finding exact solutions have some limitations. In this paper we propose an incremental heterogeneous ensemble model for time series prediction where biologically inspired algorithms offer a suitable alternative. Ensemble learning techniques are advantageously used for improving performance of various prediction methods. The quality of this kind of machine learning approaches depends on proper combination of used methods. The influence of each of the used method can change on the fly and is determined by proper choice of its weights. Finding optimal weights in prediction methods represents typical optimization problem with objective function reflecting error minimization, where biologically inspired algorithms can be used. In the proposed paper, we study several biologically inspired algorithms in the process of weights optimization. We investigate and compare ensembles using base models and ensembles optimized by biologically inspired algorithms. We demonstrate that the ensemble learning prediction models optimized by biologically inspired algorithms outperformed the base prediction methods. We present performance and accuracy results of proposed ensemble models that were evaluated on power load datasets with concept drifts.
BackT., Evolutionary Algorithms in Theory and Practice, Oxford University Press, Oxford, UK, 1996.
2.
BendtsenC., pso: Particle Swarm Optimization. R package version 1.0.3, (2012), 14.
3.
BoxG.E.P., JenkinsG.M. and ReinselG.C., Time Series Analysis: Forecasting and Control, John Wiley & Sons, New Jersey, 2008.
4.
BoxG.E.P., JenkinsG.M. and ReinselG.C., Time Series Analysis: Forecasting and Control, Holden-Day, San Francisco, CA, USA, 1970.
5.
BreimanL., FriedmanJ., StoneC.J. and OlshenR.A., Classification and Regression Trees, Chapman and Hall, 1984.
6.
CiviciogluP., Transforming geocentric cartesian coordinates to geodetic coordinates by using differential search algorithm, Comput. Geosci.46 (2012), 229-247.
7.
ClevelandR.B., ClevelandW.S., McRaeJ.E. and TerpenningI., STL: A Seasonal-Trend Decomposition Procedure Based on Loess, J. Off. Stat.6 (1990), 3-73.
8.
DorigoM., ManiezzoV. and ColorniA., Ant system: optimization by a colony of cooperating agents, IEEE Trans. Syst. Man Cybern. Part B26 (1996), 29-41.
9.
EberhartR. and KennedyJ., A new optimizer using particle swarm theory, in MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science, IEEE, 1995, pp. 9-43.
10.
Fister Jr.I., YangX.-S., FisterI., BrestJ. and FisterD., A Brief Review of Nature-Inspired Algorithms for Optimization, Elektroteh. Vestn.80 (2013), 1-7.
11.
GamaJ., liobaitėI., BifetA., PechenizkiyM. and BouchachiaA., A survey on concept drift adaptation, ACM Comput. Surv.46 (2014), 1-37.
12.
GoldbergD.E., Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley, Boston, MA, 1989.
13.
GrmanováG., RozinajováV., EzzedineA.B., LuckáM., LackoP., LódererM., VrablecováP. and LaurinecP., Application of Biologically Inspired Methods to Improve Adaptive Ensemble Learning, in Proceedings of the 7th World Congress on Nature and Biologically Inspired Computing (NaBIC2015) in Pietermaritzburg, South Africa, held December 01-03, 2015, N. Pillay, A.P. Engelbrecht, A. Abraham, M.C. du Plessis, V. Snášel, A.K. Muda, eds, Springer International Publishing, Cham, 2016, pp. 235-246.
14.
GuoL., HaoJ. and LiuM., An incremental extreme learning machine for online sequential learning problems, Neurocomputing128 (2014), 50-58.
15.
HadavandiE., GhanbariA. and Abbasian-NaghnehS., Developing a Time Series Model Based on Particle Swarm Optimization for Gold Price Forecasting, in 2010 Third International Conference on Business Intelligence and Financial Engineering, IEEE, 2010, pp. 337-340.
16.
HalimA., Hanif and IsmailI., Bio-Inspired Optimization Method: A Review, Int. J. Artif. Intell. (NNGT 2014)1 (2014), 6.
17.
HollandJ., Adaptation in Natural and Artificial Systems, MIT Press, Cambridge, MA, 1992.
18.
HongW.-C., Intelligent Energy Demand Forecasting, Springer, 2013.
ChitraA. and UmaS., An Ensemble Model of Multiple Classifiers for Time Series Prediction, Int. J. Comput. Theory Eng.2 (2010), 454-458.
21.
IkonomovskaE., GamaJ. and DžeroskiS., Learning model trees from evolving data streams, Data Min. Knowl. Discov.23 (2011), 128-168.
22.
KarabogaD. and BasturkB., A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm, J. Glob. Optim.39 (2007), 459-471.
23.
KaratzoglouA., SmolaA., HornikK. and ZeileisA., Kernlab - An S4 Package for Kernel Methods in R, J. Stat. Softw.11 (2004), 1-20.
24.
KennedyJ., EberhardR.C. and ShiY., Swarm Intelligence, Morgan Kaufmann Publishers, 2001.
25.
KennedyJ. and EberhartR., Particle swarm optimization, in Proceedings of ICNN'95 - International Conference on Neural Networks, IEEE, 1995, pp. 1942-1948.
26.
KunchevaL.I. and WhitakerC.J., Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy, Mach. Learn.51 (2003), 181-207.
27.
KvasnčkaV., PospíchalJ. and TiňoP., Evolučné algoritmy, STU Bratislava, 2000.
28.
LiawA. and WienerM., Classification and Regression by randomForest, R News2 (2002), 18-22.
29.
LimaT.P.F. and LudermirT.B., An automatic method for construction of ensembles to time series prediction, Int. J. Hybrid Intell. Syst.10 (2013), 191-203.
30.
MacArthurR.H. and WilsonE.O., The Theory of Island Biogeography, Princeton University Press, 1967.
31.
Martinez-MuozG., Hernandez-LobatoD. and SuarezA., An Analysis of Ensemble Pruning Techniques Based on Ordered Aggregation, IEEE Trans. Pattern Anal. Mach. Intell.31 (2009), 245-259.
32.
MinkuL.L., Online ensemble learning in the presence of concept drift, University of Birmingham, Birmingham, UK, 2011.
33.
MirjaliliS., MirjaliliS.M. and LewisA., Grey Wolf Optimizer, Adv. Eng. Softw.69 (2014), 46-61.
34.
Moreira-MatiasL., GamaJ., FerreiraM., Mendes-MoreiraJ. and DamasL., On predicting the taxi-passenger demand: A real-time approach, in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2013, pp. 54-65.
35.
MühlenbeinH. and Schlierkamp-VoosenD., Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization, Evol. Comput.1 (1993), 25-49.
36.
MullenK., ArdiaD., GilD., WindoverD. and ClineJ., DEoptim: An R Package for Global Optimization by Differential Evolution, J. Stat. Softw.40 (2011), 10.
37.
NauR., Moving average and exponential smoothing models, http://people.duke.edu/∼ rnau/411avg.htm, Last updated 2016, Accessed on February 23, 2016.
38.
NikumbhS., bbo: Biogeography-Based Optimization. R package version 0.2, (2014), 7.
39.
OzaN.C. and RussellS., Experimental comparisons of online and batch versions of bagging and boosting, in Proceedings of the seventh ACM SIGKDD international conference on Knowledge discovery and data mining - KDD '01, 2001, pp. 359-364.
40.
OzaN.C., Online bagging and boosting, in 2005 IEEE International Conference on Systems, Man and Cybernetics, 2005.
41.
ReidS., A Review of Heterogeneous Ensemble Methods, University of Colorado at Boulder, Boulder, CO, 2007.
42.
ShenW., BabushkinV., AungZ. and WoonW.L., An ensemble model for day-ahead electricity demand time series forecasting, in Proceedings of the the fourth international conference on Future energy systems - e-Energy '13, 2013, p. 51.
SotoJ., MelinP. and CastilloO., Time series prediction using ensembles of ANFIS models with genetic optimization of interval type-2 and type-1 fuzzy integrators, Int. J. Hybrid Intell. Syst.11 (2014), 211-226.
45.
StornR. and PriceK., Differential Evolution - A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces, J. Glob. Optim.11 (1997), 341-359.
46.
TaghaviZ.S. and SajediH., Ensemble pruning based on oblivious Chained Tabu Searches, Int. J. Hybrid Intell. Syst.12 (2016), 131-143.
47.
TherneauT., AtkinsonB. and RipleyB., rpart: Recursive Partitioning and Regression Trees. R package version 4.1-8, (2014), 34.
48.
TsoG.K.F. and YauK.K.W., Predicting electricity energy consumption: A comparison of regression analysis, decision tree and neural networks, Energy32 (2007), 1761-1768.
49.
WangY., LvJ., ZhuL. and MaY., Crystal structure prediction via particle-swarm optimization, Phys. Rev. B82 (2010), 094116.
50.
WichardJ.D. and OgorzaƚekM., Time series prediction with ensemble models, in IEEE International Conference on Neural Networks - Conference Proceedings, 2004, pp. 1625-1630.
51.
XiaoL., WangJ., HouR. and WuJ., A combined model based on data pre-analysis and weight coefficients optimization for electrical load forecasting, Energy82 (2015), 524-549.
52.
XieW., UhlmannS., KiranyazS. and GabboujM., Incremental Learning with Support Vector Data Description, in 2014 22nd International Conference on Pattern Recognition, 2014, pp. 3904-3909.
53.
YonG.V. and MuñozE., ABCoptim: Implementation of Artificial Bee Colony (ABC) Optimization. R package version 0.13.11, (2013), 5.
54.
ZangW., ZhangP., ZhouC. and GuoL., Comparative study between incremental and ensemble learning on data streams: Case study, J. Big Data1 (2014), 5.
55.
ZhangF., DaiH. and TangD., A Conjunction Method of Wavelet Transform-Particle Swarm Optimization-Support Vector Machine for Streamflow Forecasting, J. Appl. Math.2014 (2014), 1-10.
56.
ZhangP., ZhuX., ShiY., GuoL. and WuX., Robust ensemble learning for mining noisy data streams, Decis. Support Syst.50 (2011), 469-479.
57.
ZhangP., ZhuX. and ShiY., Categorizing and mining concept drifting data streams, in Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining - KDD 08, ACM Press, New York, USA, 2008, p. 812.
