This paper tackles the task of goal-conditioned dynamic manipulation of deformable objects. This task is highly challenging due to its complex dynamics (introduced by object deformation and high-speed action) and strict task requirements (defined by a precise goal specification). To address these challenges, we present Iterative Residual Policy (IRP), a general learning framework applicable to repeatable tasks with complex dynamics. IRP learns an implicit policy via delta dynamics—instead of modeling the entire dynamical system and inferring actions from that model, IRP learns delta dynamics that predict the effects of delta action on the previously observed trajectory. When combined with adaptive action sampling, the system can quickly optimize its actions online to reach a specified goal. We demonstrate the effectiveness of IRP on two tasks: whipping a rope to hit a target point and swinging a cloth to reach a target pose. Despite being trained only in simulation on a fixed robot setup, IRP is able to efficiently generalize to noisy real-world dynamics, new objects with unseen physical properties, and even different robot hardware embodiments, demonstrating its excellent generalization capability relative to alternative approaches.
BerensonD (2013) Manipulation of deformable objects without modeling and simulating deformation. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo, Japan: IEEE, pp. 4525–4532. DOI: 10.1109/IROS.2013.6697007
2.
BristowDTharayilMAlleyneA (2006) A survey of iterative learning control. IEEE Control Systems Magazine26(3): 96–114. DOI: 10.1109/MCS.2006.1636313
ChenLCZhuYPapandreouG, et al. (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: FerrariVHebertMSminchisescuC, et al. (eds), Computer Vision – ECCV 2018. Cham: Springer International Publishing, pp. 833–851.
5.
ClarkLV (1960) Effect of mental practice on the development of a certain motor skill. Research Quarterly. American Association for Health, Physical Education and Recreation31(4): 560–569.
6.
ColoméATorrasC (2018) Dimensionality reduction for dynamic movement primitives and application to bimanual manipulation of clothes. IEEE Transactions on Robotics34(3): 602–615. DOI: 10.1109/TRO.2018.2808924
7.
ColoméAPlanellsATorrasC (2015) A friction-model-based framework for reinforcement learning of robotic tasks in non-rigid environments. In: 2015 IEEE International Conference on Robotics and Automation (ICRA). Seattle, WA, USA: IEEE, pp. 5649–5654. DOI: 10.1109/ICRA.2015.7139990.
8.
HaHSongS (2021) The unreasonable effectiveness of dynamic manipulation for cloth unfolding. Conference on Robotic Learning(CoRL).
9.
HietalaJBlanco-MuleroDAlcanG, et al. (2021) Learning Visual Feedback Control for Dynamic Cloth Folding. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp.1455–1462. DOI: 10.1109/IROS47612.2022.9981376
JangirRAlenyàGTorrasC (2020) Dynamic cloth manipulation with deep reinforcement learning. In: 2020 IEEE International Conference on Robotics and Automation (ICRA). Paris, France: IEEE, pp. 4630–4636. DOI: 10.1109/ICRA40945.2020.9196659
12.
JinSRomeresDRagunathanA, et al. (2021) Trajectory optimization for manipulation of deformable objects: Assembly of belt drive units. In 2021 IEEE International Conference on Robotics and Automation (ICRA), pp. 10002–10008: IEEE.
13.
KawaharazukaKOgawaTTamuraJ, et al. (2019) Dynamic manipulation of flexible objects with torque sequence using a deep neural network. In: 2019 International Conference on Robotics and Automation (ICRA). Montreal, QC, Canada: IEEE, pp. 2139–2145. DOI: 10.1109/ICRA.2019.8793513
14.
LiYYueYXuD, et al. (2015) Folding deformable objects using predictive simulation and trajectory optimization. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany: IEEE, pp. 6000–6006. DOI: 10.1109/IROS.2015.7354231.
15.
LimVHuangHChenLY, et al. (2021) Real2Sim2Real: Self-supervised learning of physical single-step dynamic actions for planar robot casting. In 2022 International Conference on Robotics and Automation (ICRA), pp. 8282–8289. DOI: 10.1109/ICRA46639.2022.9811651
MasonMTLynchK (1993) Dynamic manipulation. Proceedings of (IROS) IEEE/RSJ International Conference on Intelligent Robots and Systems1: 152–159.
18.
McConachieDDobsonARuanM, et al. (2020) Manipulating deformable objects by interleaving prediction, planning, and control. The International Journal of Robotics Research39(8): 957–982.
19.
MooreKLJohnsonMGrimbleMJ (1993) Iterative Learning Control for Deterministic Systems. Berlin, Heidelberg: Springer-Verlag. ISBN 0387197079.
20.
NahMCKrotovARussoM, et al. (2020) Dynamic primitives facilitate manipulating a whip. In: 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob). New York, NY, USA: IEEE. pp. 685–691. DOI:10.1109/BioRob49111.2020.9224399
21.
NairAChenDAgrawalP, et al. (2017) Combining self-supervised learning and imitation for vision-based rope manipulation. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). Singapore: IEEE, pp. 2146–2153. DOI: 10.1109/ICRA.2017.7989247
22.
PathakDMahmoudiehPLuoG, et al. (2018) Zero-shot visual imitation. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Salt Lake City, UT, USA: IEEE, pp. 2131–21313. DOI:10.1109/CVPRW.2018.00278.
23.
RuggieroFLippielloVSicilianoB (2018) Nonprehensile dynamic manipulation: a survey. IEEE Robotics and Automation Letters3(3): 1711–1718. DOI: 10.1109/LRA.2018.2801939
24.
SchulmanJHoJLeeC, et al. (2016) Learning from Demonstrations through the Use of Non-rigid Registration. Cham: Springer International Publishing, pp. 339–354. ISBN 978-3-319-28872-7. URL. DOI: 10.1007/978-3-319-28872-7_20
25.
SeitaDFlorencePTompsonJ, et al. (2021) Learning to rearrange deformable cables, fabrics, and bags with goal-conditioned transporter networks. In: 2021 IEEE International Conference on Robotics and Automation (ICRA). Xi’an, China: IEEE, pp. 4568–4575. DOI: 10.1109/ICRA48506.2021.9561391
26.
SundaresanPGrannenJThananjeyanB, et al. (2020) Learning rope manipulation policies using dense object descriptors trained on synthetic depth datas. In: 2020 IEEE International Conference on Robotics and Automation (ICRA). Paris, France: IEEE, pp. 9411–9418. DOI: 10.1109/ICRA40945.2020.9197121.
27.
TabataKSekiHTsujiT, et al. (2021) Casting manipulation of unknown string by robot arm. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Prague, Czech Republic: IEEE,pp. 668–674. DOI: 10.1109/IROS51168.2021.9635837
28.
TodorovEErezTTassaY (2012) Mujoco: a physics engine for model-based control. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. Vilamoura-Algarve, Portugal: IEEE, pp. 5026–5033. DOI:10.1109/IROS.2012.6386109.
29.
XuZHeZWuJ, et al. (2020) Learning 3D Dynamic Scene Representations for Robot Manipulation. Conference on Robot Learning (CoRL).
30.
YamakawaYNamikiAIshikawaM (2011) Motion planning for dynamic folding of a cloth with two high-speed robot hands and two high-speed sliders. In: 2011 IEEE International Conference on Robotics and Automation. Shanghai, China: IEEE, pp. 5486–5491. DOI:10.1109/ICRA.2011.5979606
31.
YoshidaEAyusawaKRamirez-AlpizarIG, et al. (2015) Simulation-based optimal motion planning for deformable object. In: 2015 IEEE International Workshop on Advanced Robotics and its Social Impacts (ARSO). Lyon, France: IEEE. pp. 1–6. DOI:10.1109/ARSO.2015.7428219
32.
ZengASongSLeeJ, et al. (2019) Tossingbot: learning to throw arbitrary objects with residual physics.
33.
ZhangHIchnowskiJSeitaD, et al. (2021) Robots of the lost arc: self-supervised learning to dynamically manipulate fixed-endpoint cables. In: 2021 IEEE International Conference on Robotics and Automation (ICRA). Xi’an, China: IEEE, pp. 4560–4567.
