Sage Journals: Discover world-class research

Abstract

General-purpose clothes-folding robots do not yet exist owing to the deformable nature of textiles, making it hard to engineer manipulation pipelines or learn this task. In order to accelerate research for the learning of the robotic clothes-folding task, we introduce a video dataset of human folding demonstrations. In total, we provide 8.5 hours of demonstrations from multiple perspectives leading to 1,000 folding samples of different types of textiles. The demonstrations are recorded in multiple public places, in different conditions with a diverse set of people. Our dataset consists of anonymized RGB images, depth frames, skeleton keypoint trajectories, and object labels. In this article, we describe our recording setup, the data format, and utility scripts, which can be accessed at https://adverley.github.io/folding-demonstrations.

Keywords

Deformable objects robotic manipulation clothing learning from demonstration crowdsourcing

Get full access to this article

View all access options for this article.

References

Abbeel

(2004) Apprenticeship learning via inverse reinforcement learning. In: Proceedings of the Twenty-first International Conference on Machine Learning. New York: ACM Press.

Doumanoglou

Stria

Peleka

, et al. (2016) Folding clothes autonomously: A complete pipeline. IEEE Transactions on Robotics 32(6): 1461–1478.

English

Richardson

Garzón-Galvis

(2018) From crowdsourcing to extreme citizen science: Participatory research for environmental health. Annual Review of Public Health 39(1): 335–350.

Fang

Xie

Tai

(2017) RMPE: Regional multi-person pose estimation. In: Proceedings of the International Conference on Computer Vision (ICCV).

Finn

Levine

Abbeel

(2016) Guided cost learning: Deep inverse optimal control via policy optimization. In: International Conference on Machine Learning, pp. 49–58.

Fleming

(2019) Robots and organization studies: Why robots might not want to steal your job. Organization Studies 40(1): 23–38.

Foresti

Pellegrino

(2004) Automatic visual recognition of deformable objects for grasping and manipulation. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews) 34(3): 325–333.

Levine

Pastor

Krizhevsky

Ibarz

Quillen

(2018) Learning hand–eye coordination for robotic grasping with deep learning and large-scale data collection. The International Journal of Robotics Research 37(4–5): 421–436.

Liu

Luo

Qiu

Wang

Tang

(2016) DeepFashion: Powering robust clothes recognition and retrieval with rich annotations. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

10.

Mahler

Matl

Satish

, et al. (2019) Learning ambidextrous robot grasping policies. Science Robotics 4(26): eaau4984.

11.

Maitin-Shepard

Cusumano-Towner

Lei

Abbeel

(2010) Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding. In: 2010 IEEE International Conference on Robotics and Automation, pp. 2308–2315.

12.

Matas

James

Davison

(2018) Sim-to-real reinforcement learning for deformable object manipulation. In: Billard

Dragan

Peters

Morimoto

(eds.) Proceedings of the 2nd Conference on Robot Learning (Proceedings of Machine Learning Research, Vol. 87). PMLR, pp. 734–743. Available at: http://proceedings.mlr.press/v87/matas18a.html.

13.

Miller

van den Berg

Fritz

Darrell

Goldberg

Abbeel

(2012) A geometric approach to robotic laundry folding. The International Journal of Robotics Research 31(2): 249–267.

14.

Seita

Jamali

Laskey

, et al. (2018) Robot bed-making: Deep transfer learning using depth sensing of deformable fabric. CoRR abs/1809.09810.

15.

Sharma

Mohan

Pinto

Gupta

(2018) Multiple interactions made easy (MIME): Large scale demonstrations data for imitation. In: Conference on Robot Learning, pp. 906–915.

16.

Večerk

Hester

Scholz

, et al. (2017) Leveraging demonstrations for deep reinforcement learning on robotics problems with sparse rewards. arXiv preprint arXiv:1707.08817.

17.

Xiang

Echtler

Kerl

, et al. (2016) libfreenect2: Release 0.2. https://doi.org/10.5281/zenodo.50641.

Video dataset of human demonstrations of folding clothing for robotic folding

Abstract

Keywords

Get full access to this article

References