Deformable linear objects (DLOs), such as rods, cables, and ropes, play important roles in daily life. However, manipulation of DLOs is challenging as large geometrically nonlinear deformations may occur during the manipulation process. This problem is made even more difficult as the different deformation modes (e.g., stretching, bending, and twisting) may result in elastic instabilities during manipulation. In this paper, we formulate a physics-guided data-driven method to solve a challenging manipulation task—accurately deploying a DLO (an elastic rod) onto a rigid substrate along various prescribed patterns. Our framework combines machine learning, scaling analysis, and physical simulations to develop a physics-based neural controller for deployment. We explore the complex interplay between the gravitational and elastic energies of the manipulated DLO and obtain a control method for DLO deployment that is robust against friction and material properties. Out of the numerous geometrical and material properties of the rod and substrate, we show that only three non-dimensional parameters are needed to describe the deployment process with physical analysis. Therefore, the essence of the controlling law for the manipulation task can be constructed with a low-dimensional model, drastically increasing the computation speed. The effectiveness of our optimal control scheme is shown through a comprehensive robotic case study comparing against a heuristic control method for deploying rods for a wide variety of patterns. In addition to this, we also showcase the practicality of our control scheme by having a robot accomplish challenging high-level tasks such as mimicking human handwriting, cable placement, and tying knots.
BergouMWardetzkyMRobinsonS, et al. (2008) Discrete elastic rods. In: ACM SIGGRAPH 2008 Papers, SIGGRAPH ’08. New York, NY, USA: Association for Computing Machinery, 1–12. DOI: 10.1145/1399504.1360662.
2.
BergouMAudolyBVougaE, et al. (2010) Discrete viscous threads. ACM Transactions on Graphics29(4): 1–10.
3.
BuckhamBDriscollFRNahonM (2004) Development of a finite element cable model for use in low-tension dynamics simulation. Journal of Applied Mechanics71(4): 476–485.
4.
ChoiATongDJawedMK, et al. (2021) Implicit contact model for discrete elastic rods in knot tying. Journal of Applied Mechanics88(5): 1–13.
5.
ChoiATongDParkB, et al. (2023a) mbest: realtime deformable linear object detection through minimal bending energy skeleton pixel traversals. IEEE Robotics and Automation Letters8(8): 4863–4870.
6.
ChoiATongDTerzopoulosD, et al. (2023b) Deep Learning of Force Manifolds from the Simulated Physics of Robotic Paper Folding. Ithaca, NY: arXiv.
7.
CleggAYuWTanJ, et al. (2018) Learning to dress: synthesizing human dressing motion via deep reinforcement learning. ACM Transactions on Graphics37(6): 1–10.
8.
CrowellRHFoxRH (2012) Introduction to Knot Theory. Berlin, Germany: Springer Science & Business Media, vol. 57.
9.
EricksonZCleverHMTurkG, et al. (2018) Deep haptic model predictive control for robot-assisted dressing. In: 2018 IEEE International Conference on Robotics and Automation (ICRA). New York, NY: IEEE, 4437–4444.
GulerPPauwelsKPieropanA, et al. (2015) Estimating the deformability of elastic materials using optical flow and position-based dynamics. In: 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids). New York, NY: IEEE, 965–971.
12.
HaouchineNKuangWCotinS, et al. (2018) Vision-based force feedback estimation for robot-assisted surgery using instrument-constrained biomechanical three-dimensional maps. IEEE Robotics and Automation Letters3(3): 2160–2165.
13.
JawedMKDaFJooJ, et al. (2014) Coiling of elastic rods on rigid substrates. Proceedings of the National Academy of Sciences111(41): 14663–14668.
14.
JawedMKKhouriNKDaF, et al. (2015) Propulsion and instability of a flexible helical rod rotating in a viscous fluid. Physical Review Letters115(16): 168101.
15.
KapustaAEricksonZCleverHM, et al. (2019) Personalized collaborative plans for robot-assisted dressing via optimization and simulation. Autonomous Robots43(8): 2183–2207.
16.
KaufmannPMartinSBotschM, et al. (2009) Flexible simulation of deformable models using discontinuous galerkin fem. Graphical Models71(4): 153–167.
17.
KitaYKanehiroFUeshibaT, et al. (2011) Clothes handling based on recognition by strategic observation. In: 2011 11th IEEE-RAS International Conference on Humanoid Robots. New York, NY: IEEE, 53–58.
18.
LattanziDMillerG (2017) Review of robotic infrastructure inspection systems. Journal of Infrastructure Systems23(3): 04017004.
19.
LeeAXHuangSHHadfield-MenellD, et al. (2014) Unifying scene registration and trajectory optimization for learning from demonstrations with application to manipulation of deformable objects. In: 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York, NY: IEEE, 4402–4407.
20.
LeeRHamayaMMurookaT, et al. (2021) Sample-efficient learning of deformable linear object manipulation in the real world through self-supervision. IEEE Robotics and Automation Letters7(1): 573–580.
21.
LvNLiuJJiaY (2022) Dynamic modeling and control of deformable linear objects for single-arm and dual-arm robot manipulations. In: IEEE Transactions on Robotics. New York, NY: IEEE.
22.
MacklinMMüllerMChentanezN, et al. (2014) Unified particle physics for real-time applications. ACM Transactions on Graphics33(4): 1–12.
23.
MacklinMMüllerMChentanezN (2016) Xpbd: position-based simulation of compliant constrained dynamics. In: Proceedings of the 9th International Conference on Motion in Games. New York, NY: Association for Computing Machinery, 49–54.
24.
Maitin-ShepardJCusumano-TownerMLeiJ, et al. (2010) Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding. In: 2010 IEEE International Conference on Robotics and Automation. New York, NY: IEEE, 2308–2315.
25.
MatasJJamesSDavisonAJ (2018) Sim-to-real reinforcement learning for deformable object manipulation. Conference on Robot Learning. Zurich, Switzerland: PMLR, 734–743.
26.
McConachieDDobsonARuanM, et al. (2020) Manipulating deformable objects by interleaving prediction, planning, and control. The International Journal of Robotics Research39(8): 957–982.
27.
MillerSVan Den BergJFritzM, et al. (2012) A geometric approach to robotic laundry folding. The International Journal of Robotics Research31(2): 249–267.
28.
MitranoPMcConachieDBerensonD (2021) Learning where to trust unreliable models in an unstructured world for deformable object manipulation. Science Robotics6(54): eabd8170.
29.
MüllerMHeidelbergerBHennixM, et al. (2007) Position based dynamics. Journal of Visual Communication and Image Representation18(2): 109–118.
30.
NairAChenDAgrawalP, et al. (2017) Combining self-supervised learning and imitation for vision-based rope manipulation. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). New York, NY: IEEE, 2146–2153.
31.
PanKPhaniASGreenS (2020) Periodic folding of a falling viscoelastic sheet. Physical Review E101(1): 013002.
32.
PignatECalinonS (2017) Learning adaptive dressing assistance from human demonstration. Robotics and Autonomous Systems93: 61–75.
33.
SahaMIstoP (2007) Manipulation planning for deformable linear objects. IEEE Transactions on Robotics23(6): 1141–1150.
34.
SanchezJCorralesJABouzgarrouBC, et al. (2018) Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey. The International Journal of Robotics Research37(7): 688–716.
35.
SchulmanJGuptaAVenkatesanS, et al. (2013) A case study of trajectory transfer through non-rigid registration for a simplified suturing scenario. In: 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. New York, NY: IEEE, 4111–4117.
36.
SenSGargAGealyDV, et al. (2016) Automating multi-throw multilateral surgical suturing with a mechanical needle guide and sequential convex optimization. In: 2016 IEEE International Conference on Robotics and Automation (ICRA). New York, NY: IEEE, 4178–4185.
37.
ServinMLacoursiereC (2008) Rigid body cable for virtual environments. IEEE Transactions on Visualization and Computer Graphics14(4): 783–796.
38.
SheYWangSDongS, et al. (2021) Cable manipulation with a tactile-reactive gripper. The International Journal of Robotics Research40(12–14): 1385–1401.
39.
StefanidisDWangFKorndorfferJR, et al. (2010) Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surgical Endoscopy24(2): 377–382.
40.
SucanIAMollMKavrakiLE (2012) The open motion planning library. IEEE Robotics and Automation Magazine19(4): 72–82.
41.
SunJPengZZhouW, et al. (2015) A review on 3d printing for customized food fabrication. Procedia Manufacturing1: 308–319.
42.
SundaresanPGrannenJThananjeyanB, et al. (2020) Learning rope manipulation policies using dense object descriptors trained on synthetic depth data. 2020 IEEE International Conference on Robotics and Automation (ICRA). New York, NY: IEEE, 9411–9418.
43.
TakizawaMKudohSSuehiroT (2015) Method for placing a rope in a target shape and its application to a clove hitch. In: 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN). New York, NY: IEEE, 646–651.
44.
TangTWangCTomizukaM (2018) A framework for manipulating deformable linear objects by coherent point drift. IEEE Robotics and Automation Letters3(4): 3426–3433.
45.
TeoWERamakrishnaS (2006) A review on electrospinning design and nanofibre assemblies. Nanotechnology17(14): R89.
46.
TerzopoulosDQinH (1994) Dynamic nurbs with geometric constraints for interactive sculpting. ACM Transactions on Graphics13(2): 103–136.
47.
TimoshenkoSPGereJM (2009) Theory of Elastic Stability. Chelmsford, MA: Courier Corporation.
48.
TongDBorumAJawedMK (2021) Automated stability testing of elastic rods with helical centerlines using a robotic system. IEEE Robotics and Automation Letters7(2): 1126–1133.
49.
TongDChoiAJooJ, et al. (2023a) Snap buckling in overhand knots. Journal of Applied Mechanics90(4): 041008.
50.
TongDChoiAJooJ, et al. (2023b) A fully implicit method for robust frictional contact handling in elastic rods. Extreme Mechanics Letters58: 101924.
51.
WakamatsuHAraiEHiraiS (2006) Knotting/unknotting manipulation of deformable linear objects. The International Journal of Robotics Research25(4): 371–395.
52.
WangAKurutachTLiuK, et al. (2019) Learning Robotic Manipulation Through Visual Planning and Acting. Ithaca, NY: arXiv preprint arXiv:1905.04411.
53.
WhitcombLL (2000) Underwater robotics: out of the research laboratory and into the field. In: Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No. 00CH37065). New York, NY: IEEE, vol. 1, 709–716.
54.
YanMZhuYJinN, et al. (2020) Self-supervised learning of state estimation for manipulating deformable linear objects. IEEE Robotics and Automation Letters5(2): 2372–2379.
55.
YinHVaravaAKragicD (2021) Modeling, learning, perception, and control methods for deformable object manipulation. Science Robotics6(54): eabd8803.
56.
YuWKapustaATanJ, et al. (2017) Haptic simulation for robot-assisted dressing. In: 2017 IEEE International Conference on Robotics and Automation (ICRA). New York, NY: IEEE, 6044–6051.
57.
YuMLvKZhongH, et al. (2022) Global model learning for large deformation control of elastic deformable linear objects: an efficient and adaptive approach. In: IEEE Transactions on Robotics. New York, NY: IEEE.
58.
ZhuJNavarroBPassamaR, et al. (2019) Robotic manipulation planning for shaping deformable linear objects withenvironmental contacts. IEEE Robotics and Automation Letters5(1): 16–23.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
