Segmentation and modeling of visually symmetric objects by robot actions

Abstract

Robots usually carry out object segmentation and modeling passively. Sensors such as cameras are actuated by a robot without disturbing objects in the scene. In this paper, we present an intelligent robotic system that physically moves objects in an active manner to perform segmentation and modeling using vision. By visually detecting bilateral symmetry, our robot is able to segment and model objects through controlled physical interactions. Extensive experiments show that our robot is able to accurately segment new objects autonomously. We also show that our robot is able to leverage segmentation results to autonomously learn visual models of new objects by physically grasping and rotating them. Object recognition experiments confirm that the robot-learned models allow robust recognition. Videos of the robotic experiments are also made available.

Keywords

Autonomous computer vision fast symmetry grasping interactive learning object manipulation object recognition real time robotics segmentation SIFT

Get full access to this article

View all access options for this article.

References

Arun

Huang

Blostein

(1987) Least-squares fitting of two 3-D point sets. IEEE Transactions on Pattern Analysis and Machine Intelligence 9: 698–700.

Bouguet

(2006) Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calibdoc/ .

Boykov

Jolly

M-P

(2001) Interactive graph cuts for optimal boundary and region segmentation of objects in n-D images. In International Conference on Computer Vision (ICCV), Vol. 1, Vancouver, Canada, pp. 105–112.

Chen

(2004) Object recognition based on image sequences by using inter-feature-line consistencies. Pattern Recognition 37: 1913–1923.

Christensen

(2008) Robotics as an enabler for aging in place. In Robot Services in Aging Society IROS 2008 Workshop, Nice, France.

Elgammal

Harwood

Davis

(2000) Non-parametric model for background subtraction. In European Conference on Computer Vision, Dublin, Ireland.

Fei-Fei

Fergus

Perona

(2006) One-shot learning of object categories. IEEE Transactions on Pattern Analysis and Machine Intelligence 28: 594–611.

Fitzpatrick

(2003) First contact: an active vision approach to segmentation. In Proceedings of Intelligent Robots and Systems (IROS), Vol. 3, Las Vegas, NV. Piscataway, NJ: IEEE Press, pp. 2161–2166.

Fitzpatrick

Metta

(2003) Grounding vision through experimental manipulation. In Philosophical Transactions of the Royal Society: Mathematical, Physical, and Engineering Sciences 361: 2165–2185.

10.

Freund

Schapire

(1997) A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences 55: 119–139.

11.

Heyer

Kruglyak

Yooseph

(1999) Exploring expression data: identification and analysis of coexpressed genes. Genome Research 9: 1106–1115.

12.

Kenney

Buckley

Brock

(2009) Interactive segmentation for manipulation in unstructured environments. In Proceedings of the IEEE International Conference on Robotics and Automation, Kobe, Japan.

13.

Kim

Murphy-Chutorian

Triesch

(2006) Semi-autonomous learning of objects. In Conference on Computer Vision and Pattern Recognition Workshop, 2006 (CVPRW ’06), pp. 145–145.

14.

Kleeman

(2006a) Fast stereo triangulation using symmetry. In Australasian Conference on Robotics and Automation, Auckland, New Zealand, http://www.araa.asn.au/acra/acra2006/ .

15.

Kleeman

(2006b) Real time object tracking using reflectional symmetry and motion. In IEEE/RSJ Conference on Intelligent Robots and Systems, Beijing, China, pp. 2798–2803.

16.

Kleeman

(2008) Autonomous segmentation of near-symmetric objects through vision and robotic nudging. In International Conference on Intelligent Robots and Systems, Nice, France, pp. 3604–3609.

17.

Kleeman

(2009) Interactive learning of visually symmetric objects. In International Conference on Intelligent Robots and Systems, St Louis, MO.

18.

Zhang

Kleeman

(2008) Bilateral symmetry detection for real-time robotics applications. The International Journal of Robotics Research 27: 785–814.

19.

Lowe

(2004) Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60: 91–110.

20.

Markus

Thomas

Werner

Cremers

Horst

(2008) TVSEG—interactive total variation based image segmentation. In British Machine Vision Conference (BMVC), Leeds.

21.

Moreels

Perona

(2005) Evaluation of features detectors and descriptors based on 3D objects. In ICCV ’05: Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV–05), Vol. 1. Washington, DC, IEEE Computer Society, pp. 800–807.

22.

Mutch

Lowe

(2006) Multiclass object recognition with sparse, localized features. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1. Piscataway, NJ: IEEE Press, pp. 11–18.

23.

Pal

(1993) A review on image segmentation techniques. Pattern Recognition 26: 1277–1294.

24.

Ray

Mondada

Siegwart

(2008) What do people expect from robots? In IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France, pp. 3816–3821.

25.

Skarbek

Koschan

(1994) Colour Image Segmentation—A Survey. Technical report, Institute for Technical Informatics, Technical University of Berlin, http://citeseer.ist.psu.edu/skarbek94colour.html .

26.

Taylor

Kleeman

(2002) Grasping unknown objects with a humanoid robot. In Proceedings of Australasian Conference on Robotics and Automation, Auckland, New Zealand.

27.

Tsikos

Bajcsy

(1988) Segmentation via Manipulation. Technical Report MS-CIS-88-42, Department of Computer and Information Science, University of Pennsylvania.

28.

Ude

Omrcen

Cheng

(2008) Making object learning and recognition and active process. International Journal of Humanoid Robotics 5: 267–286.

29.

Viola

Jones

(2001) Rapid object detection using a boosted cascade of simple features. In Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai Marriott, HI.

30.

Ylä-Jääski

Ade

(1996) Grouping symmetrical structures for object segmentation and description. Computer Vision and Image Understanding 63: 399–417.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB