Small-variance asymptotics is emerging as a useful technique for inference in large-scale Bayesian non-parametric mixture models. This paper analyzes the online learning of robot manipulation tasks with Bayesian non-parametric mixture models under small-variance asymptotics. The analysis yields a scalable online sequence clustering (SOSC) algorithm that is non-parametric in the number of clusters and the subspace dimension of each cluster. SOSC groups the new datapoint in low-dimensional subspaces by online inference in a non-parametric mixture of probabilistic principal component analyzers (MPPCA) based on a Dirichlet process, and captures the state transition and state duration information online in a hidden semi-Markov model (HSMM) based on a hierarchical Dirichlet process. A task-parameterized formulation of our approach autonomously adapts the model to changing environmental situations during manipulation. We apply the algorithm in a teleoperation setting to recognize the intention of the operator and remotely adjust the movement of the robot using the learned model. The generative model is used to synthesize both time-independent and time-dependent behaviors by relying on the principles of shared and autonomous control. Experiments with the Baxter robot yield parsimonious clusters that adapt online with new demonstrations and assist the operator in performing remote manipulation tasks.
ArgallBDChernovaSVelosoMBrowningB (2009) A survey of robot learning from demonstration. Robotics and Autonomous Systems57(5): 469–483.
2.
AsfourTAzadPGyarfasFDillmannR (2008) Imitation learning of dual-arm manipulation tasks in humanoid robots. International Journal of Humanoid Robotics5(2): 183–202.
3.
BealMJGhahramaniZRasmussenCE (2002) The infinite hidden Markov model. In: DietterichTGBeckerSGhahramaniZ (eds) Advances in Neural Information Processing Systems 14. Cambridge, MA: MIT Press, pp. 577–584.
4.
BellasABouveyronCCottrellMLacailleJ (2013) Model-based clustering of high-dimensional data streams with online mixture of probabilistic PCA. Advances in Data Analysis and Classification7(3): 281–300.
5.
BillardAGCalinonSDillmannR (2016) Learning from humans. In: SicilianoBKhatibO (eds.) Handbook of Robotics. 2nd Ed.Secaucus, NJ: Springer, pp. 1995–2014.
6.
BorrelliFBemporadAMorariM (2011) Predictive Control for Linear and Hybrid Systems. Cambridge: Cambridge University Press.
7.
BouveyronCBrunetC (2014) Model-based clustering of high-dimensional data: A review. Computational Statistics and Data Analysis71: 52–78.
8.
BroderickTKulisBJordanMI (2013) MAD-Bayes: Map-based asymptotic derivations from Bayes. In: Proceedings of the 30th International Conference on Machine Learning (ICML 2013), Atlanta, GA, USA, 16–21 June 2013. pp. 226–234.
9.
BrunoDCalinonSCaldwellDG (2016) Learning autonomous behaviours for the body of a flexible surgical robot. Autonomous Robots. DOI: 10.1007/s10514-016-9544-6.
10.
CalinonS (2016) A tutorial on task-parameterized movement learning and retrieval. Intelligent Service Robotics9(1): 1–29.
11.
CalinonSD’halluinFSauserELCaldwellDGBillardAG (2010) Learning and reproduction of gestures by imitation: An approach based on hidden Markov model and Gaussian mixture regression. IEEE Robotics and Automation Magazine17(2): 44–54.
12.
CampbellTLiuMKulisBHowJPCarinL (2013) Dynamic clustering via asymptotics of the dependent Dirichlet process mixture. In: BurgesCJCBottouLGhahramaniZWeinbergerKQ (eds.) NIPS, pp. 449–457.
13.
ChenMSilvaJGPaisleyJWWangCDunsonDBCarinL (2010) Compressive sensing on manifolds using a nonparametric mixture of factor analyzers: Algorithm and performance bounds. IEEE Transactions on Signal Processing58(12): 6140–6155.
14.
DempsterAPLairdNMRubinDB (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B39(1): 1–38.
15.
DraganADSrinivasaSS (2013) A policy-blending formalism for shared control. The International Journal of Robotics Research32(7): 790–805.
16.
FigueroaNBillardA (2017) Learning complex manipulation tasks from heterogeneous and unstructured demonstrations. In: IROS Workshop on Synergies between Learning and Interaction.
17.
FoxEBSudderthEBJordanMIWillskyAS (2008) An HDP-HMM for systems with state persistence. In: Proceedings of the 25th International Conference on Machine Learning (ICML ’08), pp. 312–319.
18.
GancetJUrbinaDLetierPet al. (2015) Dexrov: Dexterous undersea inspection and maintenance in presence of communication latencies. IFAC-PapersOnLine48(2): 218–223.
19.
GancetJWeissPAntonelliGet al. (2016) Dexterous undersea interventions with far distance onshore supervision: the Dexrov project. In: IFAC Conference on Control Applications in Marine Systems (CAMS), pp. 414–419.
20.
GhahramaniZJordanMI (1994) Supervised learning from incomplete data via an EM approach. In: CowanJDTesauroGAlspectorJ (eds.) Advances in Neural Information Processing Systems, volume 6. San Francisco, CA: Morgan Kaufmann Publishers, Inc., pp. 120–127.
21.
GijsbertsAMettaG (2013) Real-time model learning using incremental sparse spectrum gaussian process regression. Neural Networks41: 59–69.
22.
HavoutisITanwaniAKCalinonS (2016) Online incremental learning of manipulation tasks for semi-autonomous teleoperation. In: IROS Workshop on Closed Loop Grasping and Manipulation.
23.
HoyosJPrietoFAlenyàGTorrasC (2016) Incremental learning of skills in a task-parameterized Gaussian mixture model. Journal of Intelligent and Robotic Systems82(1): 81–99.
24.
HugginsJHNarasimhanKSaeediAMansinghkaVK (2015) Jump-means: Small-variance asymptotics for Markov jump processes. In: Proceedings of the 32nd International Conference on Machine Learning (ICML 2015), Lille, France, 6–11 July 2015, pp. 693–701.
25.
JiangKKulisBJordanMI (2012) Small-variance asymptotics for exponential family Dirichlet process mixture models. In: PereiraFBurgesCJCBottouLWeinbergerKQ (eds.) Advances in Neural Information Processing Systems, vol. 25. Curran Associates, Inc., pp. 3158–3166.
KrishnanSGargAPatilSet al. (2015) Transition state clustering: Unsupervised surgical trajectory segmentation for robot learning. In: Proceedings of the International Symposium on Robotics Research (ISRR).
28.
KronanderKKhansariMBillardA (2015) Incremental motion learning with locally modulated dynamical systems. Robotics and Autonomous Systems70: 52–62.
29.
KulicDTakanoWNakamuraY (2008) Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden Markov chains. The International Journal of Robotics Research27(7): 761–784.
30.
KulisBJordanMI (2012) Revisiting k-means: New algorithms via Bayesian nonparametrics. In: Proceedings of the 29th International Conference on Machine Learning (ICML-12). New York, NY: ACM Press, pp. 513–520.
31.
LeeDOttC (2010) Incremental motion primitive learning by physical coaching using impedance control. In: Proceedings IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Taipei, Taiwan, pp. 4133–4140.
32.
LeeDOttCNakamuraY (2010) Mimetic communication model with compliant physical contact in human–humanoid interaction. The International Journal of Robotics Research29(13): 1684–1704.
33.
MaedaGNeumannGEwertonMLioutikovRPetersJ (2015) A probabilistic framework for semi-autonomous robots based on interaction primitives with phase estimation. In: International Symposium of Robotics Research.
34.
McLachlanGJPeelDBeanRW (2003) Modelling high-dimensional data by mixtures of factor analyzers. Computational Statistics and Data Analysis41(3-4): 379–388.
35.
NealRMHintonGE (1999) A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Learning in graphical models. Cambridge, MA: MIT Press, pp. 355–368.
36.
Nguyen-TuongDSeegerMPetersJ (2009) Model learning with local Gaussian process regression. Advanced Robotics23(15): 2015–2034.
37.
NiekumSOsentoskiSKonidarisGBartoAG (2012) Learning and generalization of complex tasks from unstructured demonstrations. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 5239–5246.
38.
OpperM (1998) A Bayesian Approach to On-line Learning. In: On-line Learning in Neural Networks. Cambridge: Cambridge University Press, pp. 363–378.
39.
PitmanJ (2002) Poisson–Dirichlet and Gem invariant distributions for split-and-merge transformations of an interval partition. Combinatorics, Probability and Computing11: 501–514.
40.
RabinerLR (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE77(2): 257–285.
41.
RoychowdhuryAJiangKKulisB (2013) Small-variance asymptotics for hidden Markov models. In: Advances in Neural Information Processing Systems, vol. 26. Curran Associates, Inc., pp. 2103–2111.
42.
SchaalSIjspeertABillardA (2003) Computational approaches to motor learning by imitation. Philosophical Transaction of the Royal Society of London: Series B, Biological Sciences358(1431): 537–547.
43.
SchaalSMohajerianPIjspeertA (2007) Dynamics systems vs. optimal control a unifying view. Progress in Brain Research165: 425–445.
44.
SethuramanJ (1994) A constructive definition of Dirichlet priors. Statistica Sinica4: 639–650.
45.
SongMWangH (2005) Highly efficient incremental estimation of Gaussian mixture models for online data stream clustering. In: Proceedings of of SPIE: Intelligent Computing - Theory and Applications III5803: 174–183.
46.
StulpFSigaudO (2015) Many regression algorithms, one unified model - A review. Neural Networks69: 60–79.
47.
SuttonRSBartoAG (1998) Reinforcement Learning: An Introduction. Cambridge, MA: MIT Press.
48.
TanwaniAK (2018) Generative Models for Learning Robot Manipulation Skills from Humans. PhD Thesis, Ecole Polytechnique Federale de Lausanne, Switzerland.
49.
TanwaniAKCalinonS (2016a) Learning robot manipulation tasks with task-parameterized semitied hidden semi-Markov model. IEEE Robotics and Automation Letters1(1): 235–242.
50.
TanwaniAKCalinonS (2016b) Online inference in Bayesian non-parametric mixture models under small variance asymptotics. In: NIPS Workshop on Advances in Approximate Bayesian Inference, pp. 1–5.
51.
TanwaniAKCalinonS (2017) A generative model for intention recognition and manipulation assistance in teleoperation. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 43–50. DOI:10.1109/IROS.2017.8202136 .
52.
TehYWJordanMIBealMJBleiDM (2006) Hierarchical Dirichlet processes. Journal of the American Statistical Association101(476): 1566–1581.
53.
TippingMEBishopCM (1999) Mixtures of probabilistic principal component analyzers. Neural Computation11(2): 443–482.
54.
VakanskiAManteghIIrishAJanabi-SharifiF (2012) Trajectory learning for robot programming by demonstration using hidden Markov model and dynamic time warping. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)42(4): 1039–1052.
55.
Van GaelJSaatciYTehYWGhahramaniZ (2008) Beam sampling for the infinite hidden Markov model. In: Proceedings of the 25th International Conference on Machine Learning (ICML ’08). New York, NY: ACM Press, pp. 1088–1095.
56.
VijayakumarSD’souzaASchaalS (2005) Incremental online learning in high dimensions. Neural Computation17(12): 2602–2634.
57.
WangYZhuJ (2015) DP-space: Bayesian nonparametric subspace clustering with small-variance asymptotics. In: Proceedings of the 32nd International Conference on Machine Learning (ICML 2015), Lille, France, 6–11 July 2015, pp. 862–870.
58.
WilsonADBobickAF (1999) Parametric hidden Markov models for gesture recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence21(9): 884–900.
ZhangZChanKLKwokJTYeungD (2004) Bayesian inference on principal component analysis using reversible jump markov chain monte carlo. In: Proceedings of the Nineteenth National Conference on Artificial Intelligence, Sixteenth Conference on Innovative Applications of Artificial Intelligence, 25–29 July 2004, San Jose, CA, pp. 372–377.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
