Sage Journals: Discover world-class research

Abstract

This paper proposes a new architecture for supervised incremental learning using neural networks. The key feature of this architecture is a special perceptron, called monitor perceptron, which decides whether a new sample belongs to a new class or to one of the known (already learnt) classes. In case if the decision by the monitor perceptron is that the sample belongs to a new class then the network is extended such that the new class is learnt by the network. The final network is a set of parallel neural networks (one for each class) whose output is fed into the monitor perceptron. A series of experiments are performed using benchmark data sets. The results obtained in these experiments are comparable with or better than those obtained using other, state of art, techniques. The growth in number of neurons is linear with respect to the growth of number of classes.

Keywords

Incremental learning neural network perceptron

Get full access to this article

View all access options for this article.

References

Al-Daoud, Incremental learning of auto-association multilayer perceptrons network, Int. Arab J. Inf. Technol. 3(1) (2006), 16–19.

Al-Daraiseh,

Kaylani,

Georgiopoulos,

Mollaghasemi,

A.S.

Wu and

Anagnostopoulos, GFAM: Evolving fuzzy ARTMAP neural networks, Neural Networks 20(8) (2007), 874–892. doi:10.1016/j.neunet.2007.05.006.

Ali and

Foroosh, Character recognition in natural scene images using rank-1 tensor decomposition, in: 2016 IEEE International Conference on Image Processing, ICIP 2016, Phoenix, AZ, USA, September 25–28, 2016, pp. 2891–2895. doi:10.1109/ICIP.2016.7532888.

G.C.

Anagnostopoulos and

Georgiopoulos, Ellipsoid ART and ARTMAP for incremental unsupervised and supervised learning, in: Proc. SPIE 4390, Applications and Science of Computational Intelligence IV, Vol. 4390, 2001, pp. 293–304. doi:10.1117/12.421180.

Aran and

Alpaydin, An incremental neural network construction algorithm for training multilayer perceptron, 2003.

Bhattacharyya,

Metla,

Bandyopadhyay,

Tudu and

Jana, Incremental PNN classifier for a versatile electronic nose, in: 2008 3rd International Conference on Sensing Technology, Nov. 2008, pp. 242–247. doi:10.1109/ICSENST.2008.4757106.

Boujelbene and

Zribi, The neural networks with an incremental learning algorithm approach for mass classification in breast cancer, Biomedical Data Mining 5 (2016), 118. doi:10.4172/2090-4924.1000118.

J.L.

Calvo-Rolle,

Fontenla-Romero,

Pérez-Sánchez and

Guijarro-Berdiòas, Adaptive inverse control using an online learning algorithm for neural networks, Informatica 25(3) (2014), 401–414. doi:10.15388/Informatica.2014.20.

G.A.

Carpenter,

Grossberg,

Markuzon,

J.H.

Reynolds and

D.B.

Rosen, Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps, IEEE Transactions on Neural Networks 3(5) (1992), 698–713. doi:10.1109/72.159059.

10.

G.A.

Carpenter,

Grossberg and

J.H.

Reynolds, ARTMAP: Supervised real-time learning and classification of nonstationary data by a self-organizing neural network, Neural Networks 4(5) (1991), 565–588. doi:10.1016/0893-6080(91)90012-T.

11.

P.M.

Ciarelli,

Oliveira and

E.O.T.

Salles, An incremental neural network with a reduced architecture, Neural Networks 35 (2012), 70–81. doi:10.1016/j.neunet.2012.08.003.

12.

P.M.

Ciarelli,

E.O.T.

Salles and

Oliveira, An evolving system based on probabilistic neural network, in: 2010 Eleventh Brazilian Symposium on Neural Networks, Oct. 2010, pp. 182–187. doi:10.1109/SBRN.2010.39.

13.

J.F.

Connolly,

Granger and

Sabourin, Incremental adaptation of fuzzy ARTMAP neural networks for video-based face classification, in: 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, July 2009, pp. 1–8. doi:10.1109/CISDA.2009.5356545.

14.

Cruz-Vega and

H.J.

Escalante, An online and incremental GRLVQ algorithm for prototype generation based on granular computing, Soft Computing (2016), 1–14. doi:10.1007/s00500-016-2042-0.

15.

T.E.

De Campos,

B.R.

Babu and

Varma, Character recognition in natural images, 2009.

16.

R.O.

Duda,

P.E.

Hart and

D.G.

Stork, Pattern Classification, 2nd edn, Wiley-Interscience, 2000. ISBN 0471056693.

17.

S.E.

Fahlman and

Lebiere, The cascade-correlation learning architecture, in: Advances in Neural Information Processing Systems, Vol. 2,

D.S.

Touretzky, ed., Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1990, pp. 524–532. ISBN 1-55860-100-7.

18.

Frean, The upstart algorithm: A method for constructing and training feedforward neural networks, Neural Comput. 2(2) (1990), 198–209. doi:10.1162/neco.1990.2.2.198.

19.

Gepperth and

Hammer, Incremental learning algorithms and applications, in: European Sympoisum on Artificial Neural Networks (ESANN), 2016.

20.

J.B.

Gomm, Adaptive neural network approach to on-line learning for process fault diagnosis, Transactions of the Institute of Measurement and Control 20(3) (1998), 144–152. doi:10.1177/014233129802000305.

21.

Grossberg, Competitive learning: From interactive activation to adaptive resonance, Cognitive Science 11 (1987), 23–63. doi:10.1111/j.1551-6708.1987.tb00862.x.

22.

F.H.

Hamker, Life-long learning cell structures – Continuously learning without catastrophic interference, Neural Networks 14(4–5) (2001), 551–573. doi:10.1016/S0893-6080(01)00018-1.

23.

G.-B.

Huang and

Chen, Letters: Convex incremental extreme learning machine, Neurocomput. 70(16–18) (2007), 3056–3062. doi:10.1016/j.neucom.2007.02.009.

24.

Kohonen, Improved versions of learning vector quantization, in: 1990 IJCNN International Joint Conference on Neural Networks, Vol. 1, June 1990, pp. 545–550. doi:10.1109/IJCNN.1990.137622.

25.

Lichman, UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences, 2013, http://archive.ics.uci.edu/ml.

26.

Mandziuk and

Shastri, Incremental class learning approach and its application to handwritten digit recognition, Inf. Sci. 141(3–4) (2002), 193–217. doi:10.1016/S0020-0255(02)00170-6.

27.

Y.L.

Murphey,

Z.H.

Chen and

L.A.

Feldkamp, An incremental neural learning framework and its application to vehicle diagnostics, Applied Intelligence 28(1) (2008), 29–49. doi:10.1007/s10489-007-0040-8.

28.

Polikar,

Upda,

S.S.

Upda and

Honavar, Learn++: An incremental learning algorithm for supervised neural networks, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 31(4) (2001), 497–508. doi:10.1109/5326.983933.

29.

Pratama,

Lu,

Anavatti,

Lughofer and

C.-P.

Lim, An incremental meta-cognitive-based scaffolding fuzzy neural network, Neurocomputing 171 (2016), 89–105. doi:10.1016/j.neucom.2015.06.022.

30.

Read,

Bifet,

Pfahringer and

Holmes, Batch-incremental versus instance-incremental learning in dynamic and evolving data, in: Proceedings of the 11th International Conference on Advances in Intelligent Data Analysis, IDA’12, Springer-Verlag, Berlin, Heidelberg, 2012, pp. 313–323. ISBN 978-3-642-34155-7. doi:10.1007/978-3-642-34156-4_29.

31.

Seera and

C.P.

Lim, Transfer learning using the online fuzzy min–max neural network, Neural Computing and Applications 25(2) (2014), 469–480. doi:10.1007/s00521-013-1517-5.

32.

Seipone and

J.A.

Bullinaria, Evolving improved incremental learning schemes for neural network systems, in: Proceedings of the IEEE Congress on Evolutionary Computation, CEC 2005, Edinburgh, UK, 2–4 September 2005, pp. 2002–2009. doi:10.1109/CEC.2005.1554941.

33.

Shen and

Hasegawa, A fast nearest neighbor classifier based on self-organizing incremental neural network, Neural Netw. 21(10) (2008), 1537–1547. doi:10.1016/j.neunet.2008.07.001.

34.

Shiotani,

Fukuda and

Shibata, A neural network architecture for incremental learning, Neurocomputing 9(2) (1995), 111–130, Control and Robotics, Part {II}. doi:10.1016/0925-2312(94)00061-V.

35.

D.F.

Specht, Probabilistic neural networks, Neural Netw. 3(1) (1990), 109–118. doi:10.1016/0893-6080(90)90049-Q.

36.

Tzikas and

Likas, An incremental Bayesian approach for training multilayer perceptrons, in: Artificial Neural Networks – ICANN 2010 – 20th International Conference, Proceedings, Part I, Thessaloniki, Greece, September 15–18, 2010, pp. 87–96. doi:10.1007/978-3-642-15819-3_12.

37.

J.H.

Wang and

H.Y.

Wang, Incremental neural network construction for text classification, in: 2014 International Symposium on Computer, Consumer and Control, June 2014, pp. 970–973. doi:10.1109/IS3C.2014.254.

38.

Xu,

Shen and

Zhao, An incremental learning vector quantization algorithm for pattern classification, Neural Computing and Applications 21(6) (2012), 1205–1215. doi:10.1007/s00521-010-0511-4.

39.

Yan,

Jiang,

Zheng,

Peng and

Li, A multilayer perceptron-based medical decision support system for heart disease diagnosis, Expert Syst. Appl. 30(2) (2006), 272–281. doi:10.1016/j.eswa.2005.07.022.

40.

K.S.

Yap,

C.P.

Lim and

Mohamad-Saleh, An enhanced generalized adaptive resonance theory neural network and its application to medical pattern classification, Journal of Intelligent and Fuzzy Systems 21(1–2) (2010), 65–78. doi:10.3233/IFS-2010-0436.

41.

Yen and

Meesad, Pattern classification by an incremental learning fuzzy neural network, in: Neural Networks, 1999. IJCNN ’99. International Joint Conference on, Vol. 5, 1999, pp. 3230–3235. doi:10.1109/IJCNN.1999.836173.

42.

Zurada, Introduction to Artificial Neural Systems, West Publishing Co., St. Paul, MN, USA, 1992. ISBN 0-314-93391-3.

INNAMP: An incremental neural network architecture with monitor perceptron

Abstract

Keywords

Get full access to this article

References