Learning new motor tasks from physical interactions is an important goal for both robotics and machine learning. However, when moving beyond basic skills, most monolithic machine learning approaches fail to scale. For more complex skills, methods that are tailored for the domain of skill learning are needed. In this paper, we take the task of learning table tennis as an example and present a new framework that allows a robot to learn cooperative table tennis from physical interaction with a human. The robot first learns a set of elementary table tennis hitting movements from a human table tennis teacher by kinesthetic teach-in, which is compiled into a set of motor primitives represented by dynamical systems. The robot subsequently generalizes these movements to a wider range of situations using our mixture of motor primitives approach. The resulting policy enables the robot to select appropriate motor primitives as well as to generalize between them. Finally, the robot plays with a human table tennis partner and learns online to improve its behavior. We show that the resulting setup is capable of playing table tennis using an anthropomorphic robot arm.
AcostaLRodrigoJMendezJMarchialGSigutM (2003) Ping-pong player prototype. IEEE Robotics and Automation Magazine10(4): 44–52.
2.
AnderssonR (1988) A Robot Ping-pong Player: Experiment in Real-time Intelligent Control. Cambridge, MA: MIT Press.
3.
ArgallBChernovaSVelosoMBrowningB (2009) A survey of robot learning from demonstration. Robotics and Autonomous System57: 469–483.
4.
BillardACalinonSDillmannRSchaalS (2008) Robot Programming by Demonstration. New York: Springer, pp. 1371–1394.
5.
BishopC (2006) Pattern Recognition and Machine Learning. New York: Springer.
6.
BishopCTippingM (2003) Bayesian Regression and Classification, volume 190. IOS Press, pp. 267–289.
7.
BitzerSHowardMVijayakumarS (2010) Using dimensionality reduction to exploit constraints in reinforcement learning. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
8.
CalinonSGuenterFBillardA (2007) On learning, representing, and generalizing a task in a humanoid robot. IEEE Transactions on Systems, Man and Cybernetics32: 286–298.
9.
ChiappaSKoberJPetersJ (2008) Using bayesian dynamical systems for motion template libraries. In: Advances in Neural Information Processing Systems (NIPS), vol. 22.
10.
FässlerHBeyerHAWenJT (1990) A robot ping pong player: optimized mechanics, high performance 3D vision, and intelligent sensor control. Robotersysteme6: 161–170.
11.
FlashTHoganN (1985) The coordination of arm movements: an experimentally confirmed mathematical model. Journal of Neurosciences5: 1688–1703.
12.
GiszterSMoxonKRybakI J C (2000) Neurobiological and neurorobotic approaches to control architectures for a humanoid motor system. Intelligent Systems and their Applications15: 64–69.
HartleyJ (1987) Toshiba progress towards sensory control in real time. The Industrial Robot14: 50–52.
15.
HashimotoHOzakiFAsanoKOsukaK (1987) Development of a ping pong robot system using 7 degrees of freedom direct drive. In: Industrial Applications of Robotics and Machine Vision (IECON), pp. 608–615.
16.
HuangYXuDTanMSuH (2011) Trajectory prediction of spinning ball for ping-pong robot. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3434–3439.
17.
IjspeertAJNakanishiJSchaalS (2002) Movement imitation with nonlinear dynamical systems in humanoid robots. In: IEEE International Conference on Robotics and Automation (ICRA).
JacobsRJordanMNowlanSHiltonG (1991) Adaptive mixures of local experts. Neural Computation3: 79–87.
20.
KnightJLoweryD (1986) Pingpong-playing robot controlled by a microcomputer. Microprocessors and Microsystems10: 332–335.
21.
KoberJMohlerBPetersJ (2008) Learning perceptual coupling for motor primitives. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
22.
KoberJMuellingKKroemerOLampertCSchölkopfBPetersJ (2010) Movement templates for learning of hitting and batting. In: IEEE International Conference on Robotics and Automation (ICRA).
23.
KoberJPetersJ (2009) Policy search for motor primitives in robotics. In: Advances in Neural Information Processing Systems (NIPS), vol. 21.
24.
KoberJWilhelmAOztopEPetersJ (2012) Reinforcement learning to adjust parametrized motor primitives to new situations. Autonomous Robots, in press.
25.
KroemerODetryRPiaterJPetersJ (2010) Grasping with vision descriptors and motor primitives. In: Proceedings of the International Conference on Informatics in Control, Automation and Robotics (ICINCO).
26.
LaiCTsayJ (2011) Self-learning for a humanoid robot ping-pong player. Advanced Robotics25: 1183–1208.
27.
LampertCPetersJ (2012) Real-time detection of colored objects in multiple camera streams with off-the-shelf hardware components. Journal of Real-Time Image Processing7: 31–41.
28.
MiyamotoHSchaalSGadolfoF. (1996) A Kendama learning robot based on bi-directional theory. Neural Networks9: 1281–1302.
29.
MiyazakiFMatsushimaMTakeuchiM (2005) Learning to dynamically manipulate: A table tennis robot controls a ball and rallies with a human being. In: Advances in Robot Control. New York: Springer, pp. 3137–341.
30.
MuellingKKoberJKroemerOPetersJ (2012) Learning to select and generalize striking movements in robot table tennis. In: Proceedings of the AAAI 2012 Fall Symposium on robots that Learn Interactively from Human Teachers.
31.
MuellingKKoberJPetersJ (2010) Learning table tennis with a mixture of motor primitives. In: 10th IEEE–RAS International Conference on Humanoid Robots (HUMANOIDS).
32.
MuellingKKoberJPetersJ (2011) A biomimetic approach to robot table tennis. Adaptive Behavior19: 359–376.
33.
NakanishiJMorimotoJEndoGChengGSchaalSKawatoM (2004) Learning from demonstration and adaption of biped locomotion. Robotics and Autonomous Systems47(2–3): 79–91.
34.
NingKKulviciusTTamosiunaiteMWörgötterF (2011) Accurate position and velocity control for trajectories based on dynamic movement primitives. In: IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, 9–13 May 2011, pp. 5006–5011.
35.
ParkDHoffmannHPastorPSchaalS (2008) Movement reproduction and obstacle avoidance with dynamic movement primitives and potential fields. In: IEEE International Conference on Humanoid Robots (Humanoids), pp. 91–98.
36.
PetersJSchaalS (2006) Policy gradient methods for robotics. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).
37.
PetersJSchaalS (2008 a) Natural actor-critic. Neurocomputing71: 1180–1190.
38.
PetersJSchaalS (2008 b) Reinforcement learning of motor skills with policy gradients. Neural Networks21: 682–697.
39.
PongasDBillardASchaalS (2005) Rapid synchronization and accurate phase-locking of rhythmic motor primitives. In: IEEE/RSJ International Conference Intelligent Robots and Systems, pp. 2911–2916.
40.
RasmussenCWilliamsC (2006) Gaussian Processes for Machine Learning. Cambridge, MA: MIT Press.
41.
SchaalS (1999) Is imitation learning the route to humanoid robots?Trends in Cognitive Sciences6: 233–242.
42.
SchaalSAtkesonCGVijayakumarS (2002) Scalable techniques from nonparameteric statistics for real-time robot learning. Applied Intelligence1: 49–60.
43.
SchaalSIjspeertABillardA (2003 a) Computational approaches to motor learning by imitation. Philosophical Transaction of the Royal Society of London: Series B, Biological Sciences358: 537–547.
44.
SchaalSMohajerianPIjspeertA (2007) Dynamics systems vs. optimal control – a unifying view. Progress in Brain Research165: 425–445.
45.
SchaalSPetersJNakanishiJIjspeertA (2003 b) Learning motor primitives. In: International Symposium on Robotics Research (ISRR).
46.
UdeAGamsAAsfourTMorimotoJ (2010) Task-specific generalization of discrete and periodic dynamic movement primitives. IEEE Transactions on Robotics26: 800–815.
47.
WangZDeisenrothMAmorHVogtDSchoelkopfBPetersJ (2012) Probabilistic modeling of human movements for intention inference. In: Proceedings of Robotics: Science and Systems (RSS).
WilliamsBToussaintMStorkeyA (2008) Modelling motion primitives and their timing in biologically executed movements. In: Advances in Neural Information Processing Systems (NIPS), vol. 20, pp. 1609–1616.
50.
ZhangZXuDTauM (2010) Visual measurements and prediction of ball trajectory for table tennis robot. IEEE Transactions on Instrumentation and Measurements59: 3195–3205.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
