Neural networks have been successfully applied for modeling time series. However, the results of long-term prediction are not satisfied. In this paper, the modified Meta-Learning is applied to the neural model. The normal Meta-Learning is modified by time-varying learning rates and adding a momentum term to improve convergence speed and robustness property. The stability of the learning process is proven. Finally, two experiments are presented to evaluate the proposed method. The first one shows an improvement in earthquakes prediction in the long-term, and the second one is a classical Benchmark problem. In both experiments, the modified Meta-Learning technique minimizes remarkably the mean square error index.
ChatfieldC., The analysis of time series: an introduction, Florida, US: CRC Press, 2004.
2.
PalitA.K. and PopovicD., Computational Intelligence in Time Series Forecasting: Theory and Engineering Applications, 2005
3.
BillingsS.A., Nonlinear System Identification: NARMAX Methods in the Time, Frequency, and Spatio-Temporal Domains, (2013-07-23).
4.
MushtaqM., AkramU., AamirM., AliH. and ZulqarnainM., Neural network techniques for time series prediction: A review, International Journal on Informatics Visualization.
5.
TaiebS.B., SorjamaaA. and BontempiG., Multiple-output modeling for multi-step-ahead time series forecasting, Neurocomputing.
6.
ClementsM.P., FransesP.H. and SwansonN.R., Forecasting economic and Financial time-series with non-linear models, International Journal of Forecasting.
7.
FarwickM., PasquazzoW., BreuR., SchwedaC., VogesK. and HanschkeI. A Meta-Model for Automated Enterprise Architecture Model Maintenance, Proceedings of the 2012 IEEE 16th International Enterprise Distributed Object Computing Conference.
8.
CuiC., HuM., WeirJ.D. and WuT., A recommendation system formeta-modeling: A meta-learning based approach, Expert Systems with Applications46 (2016), 33–44.
9.
de la RosaE. and YuW., Randomized algorithms for nonlinear systemidentification with deep learning modification, Information Sciences364–365 (2016), 197–212.
10.
GonzalezJ. and YuW., Non-linear system modeling using LSTM neuralnetworks, IFAC-PapersOnLine51(13) (2018), 485–489.
11.
YuW. and LiX., Automated Nonlinear System Modeling with MultipleFuzzy Neural Networks and Kernel Smoothing, InternationalJournal of Neural Systems20(5) (2010), 429–435.
12.
CuiC., WuT., HuM., WeirJ.D. and ChuX., Accuracy vs. robustness: bi-criteria optimizaed ensemble of metamodels, Proceedings of the 2014 Winter Simulation Conference, (2014), 616–627.
13.
VilaltaR. and DrissiY., A perspective view and survey ofmeta-learning, Artif Intell Rev18 (2002a), 77–95.
14.
BiggsJ.B., The role of meta-learning in study process, Br JEduc Psychol55 (1985), 185–212.
15.
FinnC., AbbeelP. and LevineS., Model-agnostic metalearning for fast adaptation of deep networks, In International Conference on Machine Learning, 2017, pp. 1126–1135.
16.
RaviS. and LarochelleH., Optimization as a Model for Few-Shot Learning, ICLR, (2017).
17.
MantovaniR., Rossi AlcobaçaA., VanschorenJ. and deCarvalhoA., A meta-learning recommender system for hyperparametertuning: Predicting when tuning improves SVM classifiers, Information Sciences501 (2019), 193–221.
18.
GuptaA., et al., Meta-Reinforcement Learning of Structured Exploration Strategies, NIPS, 2018.
19.
LemkeC. and BodganG., Meta-learning for time series forecastingand forecast combination, Neurocomputing73(10-12) (2010), 2006–2016.
20.
FinnC., RajeswaranA., KakadeS. and LevineS., Online meta-learning, International Conference on Machine Learning (ICML), 2019.
21.
AmasyaliM.F. and OkanK., Ersoy, A Study of Meta Learning for Regression, (2009).
22.
LemleyJ., BazrafkanS. and CorcoranP., Transfer Learning of Temporal Information for Driver Action Classification, MAICS, 2017.
23.
BrazdilP., Giraud-CarrierC. and SoaresC., Vilalta R Metalearning: applications to data mining, Springer, Berlin, (2009).
24.
NaikD.K. and MammoneR.J., Meta-Neural Networks that learn by learning, Proceedings IJCNN International Joint Conference on Neural Networks, 1992.
25.
YuW. and LiX., Discrete-time neuro identification without robustmodification, IEE Proc-Control Theory Appl Col150(3) (2003).
26.
SontagE.D., Input to State Stability: Basic Concepts and Results, In: P. Nistri, G. Stefani, (eds) Nonlinear and Optimal Control Theory, Lecture Notes in Mathematics, vol. 1932. Springer, Berlin, Heidelberg, 2008.
27.
BoxG., JenkinsG. and ReinselG., Time Series Analysis: Forecasting and Control, 4th Ed, Wiley, 2008.
28.
LimB. and ZohrenS., Time series forecasting with deep learning: A survey, arXiv preprint arXiv:2004.13408 (2020).
29.
SchoukensJ., SuykensJ. and LjungL., Wiener-Hammerstein benchmark, 15th IFAC Symposiumon System Identification, Saint-Malo, France, 2009.
30.
HospedalesT.M., AntoniouA. and MicaelliP., StorkeyMeta-LearningA.J., Meta-Learning in Neural Networks: A Survey, arXiv:2004.05439 (2020).
31.
MaS. and FildesR., Retail sales forecasting with meta-learning, European Journal of Operational Research288(1) (2020), 111–128. 10.1016/j.ejor.2020.05.038.
32.
CybenkoG., Aproximation by superposition of sigmoidal activationfunction, Math Control Signals Syst2 (1989), 857–869.
33.
NarendraK.S. and ParthasarathyK., Gradient methods for optimization of dynamical systems containing neural networks, IEEE Transactions of Neural Networks, 1991, pp. 252–262.
34.
JiangZ.P. and WangY., Input-to-State Stability for Discrete-TimeNonlinear Systems, Automatica37(2) (2001), 857–869.
35.
AsimK.M., Martínez-ÁlvarezF., BasitA. and IqbalT., Earthquake magnitude prediction in Hindukush region using machinelearning techniques, Natural Hazards85(1) (2017), 471–486.
36.
Asencio-CortésG., Martínez-ÁlvarezF., TroncosoA. and Morales-EstebanA., Medium–large earthquake magnitude prediction in Tokyo with artificial neural networks, Neural Computing andApplications28(5) (2017), 1043–1055.
37.
FloridoE., AznarteJ.L., Morales-EstebanA. and Martínez-ÁlvarezF., Earthquake magnitude prediction based onartificial neural networks: A survey, Croatian Operational Research Review7(2) (2016), 159–169.
38.
NarayanakumarS. and RajaK., A BP artificial neural network modelfor earthquake magnitude prediction in Himalayas, India, Circuits Syst7(11) (2016), 3456–3468.
39.
DengL., Three Classes of Deep Learning Architectures and Their Applications: A Tutorial Survey, 2012.
40.
MayaM. and YuW., Short-term prediction of the earthquake through Neural Networks and Meta-Learning, 16th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE19), Mexico City, Mexico, 2019.
