The ability to learn during continuous and on-line observation would be advantageous for humanoid robots, as it would enable them to learn during co-location and interaction in the human environment. However, when motions are being learned and clustered on-line, there is a trade-off between classification accuracy and the number of training examples, resulting in potential misclassifications both at the motion and hierarchy formation level. This article presents an approach enabling fast on-line incremental learning, combined with an incremental memory consolidation process correcting initial misclassifications and errors in organization, to improve the stability and accuracy of the learned motions, analogous to the memory consolidation process following motor learning observed in humans. Following initial organization, motions are randomly selected for reclassification, at both low and high levels of the hierarchy. If a better reclassification is found, the knowledge structure is reorganized to comply. The approach is validated during incremental acquisition of a motion database containing a variety of full body motions.1
Aggarwal, C.C., Han, J., Wang, J., & Yu, P. ( 2003). A framework for clustering evolving data streams. In Proceedings of the 29th VLDB Conference (pp. 81-92).
2.
Alissandrakis, A., Nehaniv, C.L., & Dautenhahn, K. (2002a). Do as I do: Correspondences across different robotic embodiments. In Proceedings of the 5th German Workshop on Artificial Life (pp. 143-152).
3.
Alissandrakis, A., Nehaniv, C.L., & Dautenhahn, K. (2002b). Imitation with Alice: Learning to imitate corresponding actions across dissimilar embodiments. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans , 32(4), 482-496.
4.
Alissandrakis, A., Nehaniv, C.L., & Dautenhahn, K. (2007). Correspondence mapping induced state and action metrics for robotic imitation. IEEE Transactions on Systems, Man, and Cybernetics, Part B, 37(2), 299-307.
5.
Bekkering, H., & Pratt, J. ( 2004). Object-based processes in the planning of goal-directed hand movements. The Quarterly Journal of Experimental Psychology, Section A, Human Experimental Psychology, 57(8), 1345-1368.
6.
Bennewitz, M., Burgard, W., Cielniak, G., & Thrun, S. ( 2005). Learning motion patterns of people for compliant robot motion. International Journal of Robotics Research, 24(1), 31-48.
7.
Bentivegna, D.C., Atkeson, C.G., & Cheng, G. ( 2006). Learning similar tasks from observation and practice . In Proceedings of the International Conference on Intelligent Robots and Systems (pp. 2677-2683).
8.
Bernardin, K., Ogawara, K., Ikeuchi, K., & Dillmann, R. ( 2005). A sensor fusion approach for recognizing continuous human grasping sequences using hidden markov models. IEEE Transactions on Robotics, 21(1), 47- 57.
9.
Billard, A., Calinon, S., & Guenter, F. ( 2006). Discriminative and adaptive imitation in uni-manual and bi-manual tasks. Robotics and Autonomous Systems, 54, 370-384.
10.
Bille, P. ( 2005). A survey on tree edit distance and related problems. Theoretical Computer Science, 337, 217-239.
11.
Breazeal, C., Berlin, M., Brooks, A., Gray, J., & Thomaz, A.L. ( 2006). Using perspective taking to learn from ambiguous demonstrations . Robotics and Autonomous Systems, 54, 385-393.
12.
Breazeal, C., & Scassellati, B. (2002). Robots that imitate humans . Trends in Cognitive Sciences, 6(11), 481-487.
13.
Byrne, R.W. ( 1999). Imitation without intentionality. using string parsing to copy the organization of behavior. Animal Cognition, 2, 63-72.
14.
Byrne, R.W., & Russon, A.E. ( 1998). Learning by imitation: a hierarchical approach. Behavioral and Brain Sciences, 21, 667-721.
15.
Calinon, S., & Billard, A. ( 2007a). Active teaching in robot programming by demonstration . In Proceedings of the IEEE International Conference on Robot and Human Interactive Communication (pp. 702-707).
16.
Calinon, S., & Billard, A. ( 2007b). Incremental learning of gestures by imitation in a humanoid robot. In Proceedings of the ACM/IEEE International Conference on Human- Robot Interaction (pp. 255-262).
17.
Calinon, S., & Billard, A. ( 2007c). Learning of gestures by imitation in a humanoid robot . In C. L. Nehaniv & K. Dautenhahn (Eds.), Imitation and Social Learning in Robots, Humans and Animals (pp. 153-177). Cambridge University Press.
18.
Calinon, S., Guenter, F., & Billard, A. ( 2007). On learning, representing and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics, Part B, 37(2), 286-298.
19.
Chaminade, T., Meltzoff, A.N., & Decety, J. ( 2002). Does the end justify the means? a pet exploration of the mechanisms involved in human imitation. NeuroImage, 15, 318- 328.
20.
Dehaene, S., Changeux, J.-P., & Nadal, J.P. ( 1987). Neural networks that learn temporal sequences by selection . Proceedings of the National Academy of Sciences of the USA , 84, 2727-2731.
21.
Demiris, Y., & Khadhouri, B. ( 2006). Hierarchical attentive multiple models for execution and recognition of actions. Journal of Robotics and Autonomous Systems, 54, 361- 369.
22.
Diekelmann, S., & Born, J. ( 2007). One memory, two ways to consolidate? Nature Neuroscience, 10(9), 1085-1086.
23.
Dillmann, R. ( 2004). Teaching and learning of robot tasks via observation of human performance. Journal of Robotics and Autonomous Systems , 47, 109-116.
24.
Dillmann, R., Rogalla, O., Ehrenmann, M., Zollner, R., & Bordegoni, M. ( 1999). Learning robot behaviour and skills based on human demonstration and advice: The machine learning paradigm. In Proceedings of the International Symposium on Robotics Research (pp. 229-238).
25.
Dominey, P.F., Metta, G., Nori, F., & Natale, L. ( 2008). Anticipation and initiative in human-humanoid interaction . In Proceedings of the IEEE International Conference on Humanoid Robots (pp. 693-699).
26.
Driskell, J.E., Copper, C., & Moran, A. ( 1994). Does mental practice enhance performance? Journal of Applied Psychology, 79(4), 481-492.
27.
Ekvall, S., Aarno, D., & Kragic, D. ( 2006). Online task recognition and real-time adaptive assistance for computer-aided machine control. IEEE Transactions on Robotics , 22(5), 1029-1033.
28.
Erlhagen, W., Mukovskiy, A., Bicho, E., Panin, G., Kiss, C., Knoll, A., van Schie, H., & Bekkering, H. ( 2006). Goal-directed imitation for robots: A bio-inspired approach to action understanding and skill learning. Robotics and Autonomous Systems, 54, 353-360.
29.
Erlhagen, W., Mukovskiy, A., Chersi, F., & Bicho, E. ( 2007). On the development of intention understanding for joint action tasks. In Proceedings of the 6th IEEE International Conference on Development and Learning (pp. 140-145).
Gallese, V., Fadiga, L., Rogassi, L., & Rizzolatti, G. (1996). Action recognition in the premotor cortex. Brain, 119, 593-609.
32.
Hayes, S.J., Ashford, D., & Bennett, S.J. ( 2008). Goal-directed imitation: The means to an end. Acta Psychologica, 127, 407-415.
33.
Heyes, C. ( 2001). Causes and consequences of imitation. Trends in Cognitive Sciences, 5(6), 253-261.
34.
Heyes, C., & Ray, E. ( 2000). What is the significance of imitation in animals? Advances in the Study of Behavior, 29, 215-245.
35.
Ho, M.A.T., Yamada, Y., & Umetani, Y. ( 2005). An adaptive visual attentive tracker for human communicational behaviors using HMM-based TD learning with new state distinction capability . IEEE Transactions on Robotics, 21(3), 497-504.
36.
Iba, S., Paredis, C.J.J., & Khosla, P.K. ( 2005). Interactive multi-modal robot programming. International Journal of Robotics Research, 24(1), 83-104.
37.
Inamura, T., Toshima, I., Tanie, H., & Nakamura, Y. ( 2004). Embodied symbol emergence based on mimesis theory. The International Journal of Robotics Research, 23(4-5), 363-377.
38.
Jackson, P.L., Lafleur, M.F., Malouin, F., Richards, C., & Doyon, J. ( 2001). Potential role of mental practice using motor imagery in neurologic rehabilitation. Archives of Physical Medicine and Rehabilitation, 82, 1133-1141.
39.
Jain, A.K., Murty, M.N., & Flynn, P.J. ( 1999). Data clustering: A review. ACM Computing Surveys, 31(3), 264-323.
40.
Jeannerod, M. ( 1995). Mental imagery in the motor context. Neuropsychologia, 33(11), 1419-1432.
41.
Jenkins, O.C., & Matarić, M. (2004). A spatio-temporal extension to isomap nonlinear dimension reduction. In Proceedings of the International Conference on Machine Learning (pp. 441-448).
42.
Kadone, H., & Nakamura, Y. ( 2005). Symbolic memory for humanoid robots using hierarchical bifurcations of attractors in nonmonotonic neural networks. In Proceedings of the International Conference on Intelligent Robots and Systems (pp. 2900-2905).
43.
Kang, S.B., & Ikeuchi, K. ( 1993). Toward automatic robot instruction from perception - temporal segmentation of tasks from human hand motion. IEEE Transactions on Robotics and Automation, 11, 432-443.
44.
Keysers, C., & Perrett, D.I. ( 2004). Demystifying social cognition: a Hebbian perspective . Trends in Cognitive Sciences, 8(11), 501-507.
45.
Kragic, D., Marayong, P., Li, M., Okamura, A.M., & Hager, G.D. ( 2005). Human-machine collaborative systems for microsurgical applications. International Journal of Robotics Research, 24(9), 731-742.
46.
Krakauer, J.W., & Shadmehr, R. ( 2006). Consolidation of motor memory. Trends in Neurosciences, 29(1), 58-64.
47.
Krueger, V., Kragic, D., Ude, A., & Geib, C. ( 2007). The meaning of action: A review on action recognition and mapping. Advanced Robotics, 21(13), 1473-1501.
48.
Kulić, D., Lee, D., Ott, C., & Nakamura, Y. ( 2008). Incremental learning of full body motion primitives for humanoid robots. In Proceedings of the IEEE International Conference on Humanoid Robots (pp. 326-332).
49.
Kulić, D., & Nakamura, Y. ( 2008a). Incremental learning and memory consolidation of whole body motion patterns. In Proceedings of the International Conference on Epigenetic Robotics (pp. 61-68).
50.
Kulić, D., & Nakamura, Y. ( 2008b). Scaffolding on-line segmentation of full body human motion patterns. In Proceedings of the IEEE/RJS International Conference on Intelligent Robots and Systems (pp. 2860-2866).
51.
Kulić, D., Takano, W., & Nakamura, Y. ( 2007a). Incremental learning of full body motions via adaptive factorial hidden markov models. In Proceedings of the International Conference on Epigenetic Robotics (pp. 69-76).
52.
Kulić, D., Takano, W., & Nakamura, Y. ( 2007b). Representability of human motions by factorial hidden markov models. In Proceedings of the IEEE International Conference on Intelligent Robots and Systems (pp. 2388-2393).
53.
Kulić, D., Takano, W., & Nakamura, Y. ( 2008a). Combining automated on-line segmentation and incremental clustering for whole body motions. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 2591-2598).
54.
Kulić, D., Takano, W., & Nakamura, Y. ( 2008b). Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden markov chains. International Journal of Robotics Research, 27(7), 761-784.
55.
Kuniyoshi, Y., Inaba, M., & Inoue, H. ( 1989). Teaching by showing: Generating robot programs by visual observation of human performance. In Proceedings of the International Symposium on Industrial Robots (pp. 119-126).
56.
Kuniyoshi, Y., & Inoue, H. ( 1994). Learning by watching: Extracting reusable task knowledge from visual observation of human performance. IEEE Transactions on Robotics and Automation, 10(6), 799-822.
57.
Kurihara, K., Hoshino, S., Yamane, K., & Nakamura, Y. ( 2002). Optical motion capture system with pan-tilt camera tracking and realtime data processing. In Proceedings of the IEEE International Conference on Robotics and Automation, (Vol. 2, pp. 1241-1248).
58.
Lee, D., & Nakamura, Y. ( 2005). Mimesis from partial observations. In Proceedings of the International Conference on Intelligent Robots and Systems (pp. 1911-1916).
59.
Liu, K.P., Chan, C.C., Lee, T.M., & Hui-Chan, C.W. (2004). Mental imagery for promoting relearning for people after stroke: A randomized controlled trial. Archives of Physical Medicine and Rehabilitation, 85, 1403-1408.
60.
Liu, S., & Asada, H. ( 1992). Transferring manipulative skills to robots: representation and acquisition of tool manipulative skills using a process dynamics model . Journal of Dynamic Systems, Measurement and Control, 114(2), 220-228.
61.
Lockerd, A., & Breazeal, C. ( 2004). Tutelage and socially guided robot learning. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 3475-3480).
62.
McClelland, J.L., McNaughton, B.L., & O’Reilly, R.C. (1995). Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102(3), 419-457.
63.
Meltzoff, A.N. ( 2002). Imitation as a mechanism of social cognition: origins of empathy, theory of mind, and the representation of action. In U. Goswami (Ed.), Blackwell handbook of childhood cognitive development (pp. 6-25). Blackwell Publishers.
64.
Meltzoff, A.N. ( 2005). Imitation and other minds: the "like me" hypothesis. In S. Hurley & N. Chater (Eds.), Perspectives on imitation: from neuroscience to social science (Vol. 2, pp. 55-77). Cambridge, MA: MIT Press.
65.
Nakanishi, J., Morimoto, J., Endo, G., Cheng, G., Schaal, S., & Kawato, M. ( 2004). Learning from demonstration and adaptation of biped locomotion . Robotics and Autonomous Systems, 47, 79-91.
66.
Mussa-Ivaldi, F.A., Hogan, N., & Bizzi, E. ( 1985). Neural, mechanical, and geometric factors subserving arm posture in humans. The Journal of Neuroscience, 5(10), 2732-2743.
67.
Nicolescu, M.N., & Matarić, M.J. (2001). Learning and interacting in human-robot domains. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 31(5), 419-430.
68.
Nicolescu, M.N., & Matarić, M.J. (2005). Task learning through imitation and human-robot interaction. In K. Dautenhahn & C. Nehaniv (Eds.), Imitation and social learning in robots, humans and animals: behavioral, social and communicative dimensions. Cambridge University Press .
69.
Ogata, T., Sugano, S., & Tani, J. ( 2005). Open-end human-robot interaction from the dynamical systems perspective: mutual adaptation and incremental learning. Advanced Robotics, 19, 651-670.
70.
Oztop, E., & Arbib, M.A. ( 2002). Schema design and implementation of the grasp-related mirror neuron system. Biological Cybernetics, 87, 116-140.
71.
Payne, J.D., & Nadel, L. ( 2004). Sleep, dreams, and memory consolidation: The role of the stress hormone cortisol. Learning and Memory, 11, 671-678.
72.
Petrovskaya, A., & Ng, A. ( 2007). Probabilistic mobile manipulation in dynamic environments, with application to opening doors. In Proceedings of the International Joint Conference on Artificial Intelligence (pp. 2178-2184).
73.
Rabiner, L.R. ( 1989). A tutorial on hidden markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257-286.
74.
Rizzolatti, G., & Craighero, L. ( 2004). The mirror-neuron system. Annual Reviews of Neuroscience, 27, 169-192.
75.
Rizzolatti, G., Fogassi, L., & Gallese, V. ( 2001). Neurophysical mechanisms underlying the understanding and imitation of action. Nature Reviews: Neuroscience, 2, 661- 670.
76.
Rodrigues, P.P., Gama, J., & Pedroso, J.P. ( 2008). Hierarchical clustering of time series data streams. IEEE Transactions on Knowledge and Data Engineering. To Appear.
77.
Rosenbaum, D., Meulenbroek, R., Jansen, C., & Vaughan, J. ( 2001). Posture-based motion planning: Applications to grasping . Psychological Review, 108(4), 709-734.
78.
Rosenbaum, D., Meulenbroek, R., & Vaughan, J. ( 2001). Planning reaching and grasping movements: Theoretical outlines and practical implications. Motor Control, 5(2), 99- 115.
79.
Schaal, S., Ijspeert, A., & Billard, A. ( 2003). Computational approaches to motor learning by imitation . Philosophical Transactions of the Royal Society of London B: Biological Sciences, 358, 537-547.
80.
Shadmehr, R., & Holcomb, H.H. ( 1997). Neural correlates of motor memory consolidation. Science, 277, 821-825.
81.
Shasha, D., & Zhang, K. ( 1997). Approximate tree pattern matching. In A. Apostolico & Z. Ghalil (Eds.), Pattern Matching (pp. 341-371). Oxford University Press.
82.
Startner, T., & Pentland, A. ( 1995). Visual recognition of American sign language using hidden markov models. In Proceedings of the International Conference on Automatic Face and Gesture Recognition (pp. 189-194).
83.
Stickgold, R. ( 2005). Sleep-dependent memory consolidation. Nature, 437, 1272-1278.
84.
Takano, W., Tanie, H., & Nakamura, Y. ( 2005). Key feature extraction for probabilistic categorization of human motion patterns. In Proceedings of the IEEE International Conference on Advanced Robotics (pp. 424-430).
85.
Takano, W., Yamane, K., Sugihara, T., Yamamoto, K., & Nakamura, Y. ( 2006). Primitive communication based on motion recognition and generation with hierarchical mimesis model. In Proceedings of the IEEE International Conference on Robotics and Automation (pp. 3602-3608).
86.
Taylor, G.W., Hinton, G.E., & Roweis, S. ( 2006). Modeling human motion using binary latent variables. In Proceedings of the Conference on Neural Information Processing Systems (pp. 1345-1352).
87.
Wohlschlaeger, A., Gattis, M., & Bekkering, H. ( 2003). Action generation and action perception in imitation: An instance of the ideomotor principle. Philosophical Transactions of the Royal Society of London B, 358, 501-515.
88.
Yang, J., Xu, Y., & Chen, C.S. ( 1997). Human action learning via hidden markov model. IEEE Transactions on Systems, Man and Cybernetics - Part A: Systems and Humans , 27(1), 34-44.
89.
Yue, G., & Cole, K.J. ( 1992). Strength increases from the motor program: comparison of training with maximal voluntary and imagined muscle contractions. Journal of Neurophysiology, 67, 1114-1123.
90.
Zhang, T., Ramakrishnan, R., & Livny, M. ( 1996). Birch: An efficient data clustering method for very large databases. In Proceedings of the ACM SIGMOD (pp. 103-114).
