Performing robust goal-directed manipulation tasks remains a crucial challenge for autonomous robots. In an ideal case, shared autonomous control of manipulators would allow human users to specify their intent as a goal state and have the robot reason over the actions and motions to achieve this goal. However, realizing this goal remains elusive due to the problem of perceiving the robot’s environment. We address and describe the problem of axiomatic scene estimation for robot manipulation in cluttered scenes which is the estimation of a tree-structured scene graph describing the configuration of objects observed from robot sensing. We propose generative approaches to scene inference (as the axiomatic particle filter, and the axiomatic scene estimation by Markov chain Monte Carlo based sampler) of the robot’s environment as a scene graph. The result from AxScEs estimation are axioms amenable to goal-directed manipulation through symbolic inference for task planning and collision-free motion planning and execution. We demonstrate the results for goal-directed manipulation of multi-object scenes by a PR2 robot.
AbbeelPNgAY (2004) Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the twenty-first international conference on machine learning, ICML ‘04, New York, USA, p.1. ACM.
2.
AkgunBCakmakMJiangK. (2012) Keyframe-based learning from demonstration. International Journal of Social Robotics4(4): 343–355.
3.
AtkesonCGSchaalS (1997) Robot learning from demonstration. In: Proceedings of the fourteenth international conference on machine learning, ICML ‘97, San Francisco, CA, pp.12–20. Morgan Kaufmann Publishers Inc.
4.
BiswasJVelosoMM (2013) Localization and navigation of the cobots over long-term deployments. International Journal of Robotics Research32(14): 1679–1694.
5.
CalinonSBillardA (2007) Incremental learning of gestures by imitation in a humanoid robot. In: Second conference on human-robot interaction (HRI), Arlington, VA, pp. 255–262. IEEE.
6.
ChaoCCakmakMThomazAL (2011) Towards grounding concepts for transfer in goal learning from demonstration. In: Proceedings of the joint IEEE international conference on development and learning and on epigenetic robotics (ICDL-EpiRob). Frankfurt, Germany, pp. 1–6. IEEE.
7.
ChernovaSVelosoM (2009) Interactive policy learning through confidence-based autonomy. Journal of Artificial Inelligence Research34(1): 1–25.
8.
ChoiCChristensenHI (2013) RGB-d object tracking: A particle filter approach on GPU. In: Intelligent robots and systems (IROS), 2013 IEEE/RSJ international conference on, Tokyo, Japan, pp.1084–1091. IEEE.
9.
CiocarlieMHsiaoKJonesEG. (2014) Towards reliable grasping and manipulation in household environments. In: Experimental Robotics. Heidelberg: Springer Berlin, pp.241–252.
10.
ColletABerensonDSrinivasaSS. (2009) Object recognition and full pose registration from a single image for robotic manipulation. In: IEEE international conference on robotics and automation (ICRA), Kobe, Japan. pp. 48–55. IEEE.
11.
ColletAMartinezMSrinivasaSS (2011) The moped framework: Object recognition and pose estimation for manipulation. International Journal of Robotics Research30(10): 1284–1306.
12.
CoradeschiSSaffiottiA (2003) An introduction to the anchoring problem. Robotics and Autonomous Systems43(2): 85–96.
13.
CosgunAHermansTEmeliV. (2011) Push planning for object placement on cluttered table surfaces. In: 2011 IEEE/RSJ international conference on intelligent robots and systems, San Francisco, CA, pp.4627–4632. IEEE.
14.
CrickCOsentoskiSJayG. (2011) Human and robot perception in large-scale learning from demonstration. In: Proceedings of the 6th ACM/IEEE international conference on human-robot interaction (HRI). New York, NY, pp. 339–346. ACM.
15.
DellaertFFoxDBurgardWThrunS (1999) Monte carlo localization for mobile robots. In: IEEE international conference on robotics and automation. Detroit, MI, pp. 1322–1328. IEEE.
16.
DesinghKJenkinsOCReveretL. (2016) Physically plausible scene estimation for manipulation in clutter. In: IEEE-RAS international conference on humanoid robots (Humanoids 2016). 15–17 November, Cacun, Mexico.
17.
DogarMSrinivasaS (2011) A framework for push-grasping in clutter. In: Proceedings of robotics: Science and systems, Los Angeles, CA, pp. 65–72.
18.
DogarMRHsiaoKCiocarlieMT.(2012) Physics-based grasp planning through clutter. In: Robotics: Science and systems. p. 504. MIT Press.
19.
FallonMJohannssonHLeonardJ (2012) Efficient scene simulation for robust monte carlo localization using an rgb-d camera. In: Robotics and automation (ICRA), 2012 IEEE international conference on, St. Paul, MN, pp.1663–1670. IEEE.
20.
FelzenszwalbPFGirshickRBMcAllesterD. (2010) Object detection with discriminatively trained part-based models. IEEE Transactions on Pattern Analysis and Machine Intelligence32(9): 1627–1645.
21.
FikesRENilssonNJ (1972) Strips: A new approach to the application of theorem proving to problem solving. Artificial intelligence2(3): 189–208.
22.
GoodrichMAJr DROCrandallJW. (2001) Experiments in adjustable autonomy. In: Proceedings of the IJCAI workshop on autonomy, delegation and control: Interacting with intelligent agents, Seattle, WA, pp.1624–1629. IJCAI.
23.
GrollmanDHJenkinsOC (2008) Sparse incremental learning for interactive robot control policy estimation. In: International conference on robotics and automation (ICRA 2008), 19–23 May, Pasadena, CA, pp.3315–3320. IEEE.
24.
HarnadS (1990) The symbol grounding problem. Physica D: Nonlinear Phenomena42(1): 335–346.
25.
HartSDinhPHambuchenK (2015) The affordance template ros package for robot task programming. In: Robotics and automation (ICRA), 2015 IEEE International conference on, Seattle, WA pp.6227–6234. IEEE.
26.
HastingsWK (1970) Monte carlo sampling methods using markov chains and their applications. Biometrika57(1): 97–109.
27.
HerbstERenXFoxD (2011) Rgb-d object discovery via multi-scene analysis. In: Intelligent robots and systems (IROS), 2011 IEEE/RSJ international conference on, San Francisco, CA, pp.4850–4856. IEEE.
28.
JenkinsOCMatarićMJ (2004) Performance-derived behavior vocabularies: Data-driven acqusition of skills from motion. International Journal of Humanoid Robotics1(2): 237–288.
29.
JohoDTipaldiGDEngelhardN. (2012) Nonparametric bayesian models for unsupervised scene analysis and reconstruction. In: Proceedings of robotics: Science and systems, Sydney, Australia. pp. 161–168.
30.
KaelblingLPLittmanMLCassandraAR (1998) Planning and acting in partially observable stochastic domains. Artificial Intelligence101(1-2): 99–134.
31.
KaelblingLPLozano-PérezT (2013) Integrated task and motion planning in belief space. The International Journal of Robotics Research32(9–10): 1194–1227. Doi: 10.1177/0278364913484072.
32.
KempCCAndersonCDNguyenH. (2008) A point-and-click interface for the real world: Laser designation of objects for mobile manipulation. In: Human-robot interaction (HRI), 2008 3rd ACM/IEEE international conference on, 12–15 March, Amsterdam, The Netherlands, pp.241–248. IEEE.
33.
KirkJRLairdJE (2013) Learning task formulations through situated interactive instruction. In: Proceedings of the second annual conference on advances in cognitive systems, 12–14 December, Baltimore, Maryland, pp.219–236.
34.
KoberJPetersJ (2011) Policy search for motor primitives in robotics. Machine Learning84(1-2): 171–203.
35.
KuipersB (2000) The spatial semantic hierarchy. Artificial Intelligence119(1): 191–233.
36.
LairdJENewellARosenbloomPS (1987) Soar: An architecture for general intelligence. Artificial Intelligence33: 1–64.
37.
LangTToussaintMKerstingK (2012) Exploration in relational domains for model-based reinforcement learning. Journal of Machine Learning Research13(1): 3725–3768.
38.
LiuZChenDWurmKM. (2015) Table-top scene analysis using knowledge-supervised mcmc. Robotics and Computer-Integrated Manufacturing33: 110–123.
39.
ManfredottiCEFleetDJ SZ. (2010) Relational particle filtering. In: Monte Carlo methods for modern applications, 2010 NIPS workshop, Whistler, Canada.
40.
McDermottDGhallabMHoweA. (1998) Pddl - the planning domain definition language. Technical Report TR-98-003, Yale Center for Computational Vision and Control, USA.
MohanSMiningerAHKirkJR. (2012) Acquiring grounded representations of words with situated interactive instruction. In: Advances in Cognitive Systems pp. 113–130. Citeseer.
43.
NarayanaswamySBarbuASiskindJM (2011) A visual language model for estimating object pose and structure in a generative visual domain. In: Robotics and automation (ICRA), 2011 IEEE international conference on, Shanghai, China, pp.4854–4860. IEEE.
44.
NicolescuMMatarićMJ (2003) Natural methods for robot task learning: Instructive demonstration, generalization and practice. In: 2nd international joint conference on autonomous agents and multi-agent systems (AAMAS), 14–18 July, Melbourne, Australia, pp.241–248.
45.
PapazovCHaddadinSParuselS. (2012) Rigid 3d geometry matching for grasping of known objects in cluttered scenes. The International Journal of Robotics Research31(4): 538–553. Doi: 10.1177/0278364911436019.
46.
ParkerSGBiglerJDietrichA. (2010) Optix: A general purpose ray tracing engine. ACM Transactions on Graphics29(4): 66:1–66:13. Doi: 10.1145/1778765.1778803.
47.
PasATPlattR (2014) Localizing handle-like grasp affordances in 3d point clouds. In: International symposium on experimental robotics (ISER), pp. 623–638. Springer International Publishing.
48.
PastorPKalakrishnanMChittaS. (2011) Skill learning and task outcome prediction for manipulation. In: Proceedings of the 2011 IEEE international conference on robotics & automation (ICML 2011), Shanghai, China, pp. 3828–3834. IEEE.
49.
PlattRJFaggAHGrupenRA (2010) Null-space grasp control: Theory and experiments. IEEE Transactions on Robotics26(2): 282–295.
50.
QuigleyMConleyKGerkeyBP. (2009) Ros: an open-source robot operating system. In: ICRA workshop on open source software, Kobe, Japan, volume 3, p. 5. ICRA.
51.
RosmanBRamamoorthyS (2011) Learning spatial relationships between objects. International Journal on Robotics Research30(11): 1328–1342.
52.
RusuRBMartonZCBlodowN. (2008a) Towards 3D point cloud based object maps for household environments. Robotics and Autonomous Systems56(11): 927–941.
53.
RusuRBMartonZCBlodowN. (2008b) Towards 3d point cloud based object maps for household environments. Robotics and Autonomous Systems56(11): 927–941.
54.
NiekumS GKOsentoskiSBartoAG (2012) Learning and generalization of complex tasks from unstructured demonstrations. Intelligent Robots and Systems, Vilamoura, Portugal, pp. 5239–5246. IEEE.
55.
SaitoTTakahashiT (1990) Comprehensible rendering of 3-d shapes. In: Proceedings of the 17th annual conference on computer graphics and interactive techniques, SIGGRAPH ‘90, New York, USA, pp.197–206. ACM.
56.
ShreinerDSellersGKessenichJLicea-KaneB (2013) OpenGL programming guide: The official guide to learning OpenGL, Version 4.3, 8th ed, Upper Saddle River, NJ. Addison-Wesley Professional.
57.
SmartWDKaelblingLP (2002) Effective reinforcement learning for mobile robots. In: Robotics and automation, 2002. Proceedings. ICRA ‘02. IEEE international conference on, volume 4, Washington, DC, pp.3404–3410. IEEE.
58.
SrivastavaSRianoLRussellS. (2013) Using classical planners for tasks with continuous operators in robotics. In: Proceedings of the ICAPS workshop on planning and robotics (PlanRob), Beijing, China.
59.
SuiZJenkinsOCDesinghK (2015) Axiomatic particle filtering for goal-directed robotic manipulation. In: International conference on intelligent robots and systems (IROS 2015), 28 September–2 October, Hamburg, Germany, pp.4429–4436.
60.
TavenrathMKubischC (2013) Advanced scenegraph rendering pipeline. In: GPU technology conference, March, San Jose, CA.
61.
TenorthMBeetzM (2013) Knowrob: A knowledge processing infrastructure for cognition-enabled robots. International Journal of Robotics Research32(5): 566–590.
62.
TraftonGHiattLHarrisonA. (2013) Act-r/e: An embodied cognitive architecture for human-robot interaction. Journal of Human-Robot Interaction2(1): 30–55.
63.
VondrakMSigalLHodginsJ. (2012) Video-based 3D motion capture through biped control. ACM Transaction of Graphics (TOG) (Proceedings of ACM SIGGRAPH)New York, NY, pp. 27:1–27:12. ACM.
WintermuteSLairdJE (2008) Bimodal spatial reasoning with continuous motion. In: Proceedings of the 23rd national conference on artificial intelligence - Volume 3, AAAI’08, pp.1331–1337. AAAI Press.
66.
YancoHNortonAOberW. (2015) Analysis of human-robot interaction at the darpa robotics challenge trials. Journal of Field Robotics32(3): 420–444.
67.
ZettlemoyerLSPasulaHMKaelblingLP (2007) Logical particle filtering. In: Dagstuhl seminar on probabilistic, logical and relational learning, Dagstuhl, Germany. IBFI.
68.
ZhangKShashaD (1989) Simple fast algorithms for the editing distance between trees and related problems. SIAM Journal on Computing18(6): 1245–1262.
69.
ZhangLTrinkleJ (2012) The application of particle filtering to grasping acquisition with visual occlusion and tactile sensing. In: Robotics and automation (ICRA), 2012 IEEE international conference on, St. Paul, MN, pp.3805–3812. IEEE.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
