Integrating visual perception and manipulation for autonomous learning of object representations

Abstract

Humans can effortlessly perceive an object they encounter for the first time in a possibly cluttered scene and memorize its appearance for later recognition. Such performance is still difficult to achieve with artificial vision systems because it is not clear how to define the concept of objectness in its full generality. In this paper we propose a paradigm that integrates the robot’s manipulation and sensing capabilities to detect a new, previously unknown object and learn its visual appearance. By making use of the robot’s manipulation capabilities and force sensing, we introduce additional information that can be utilized to reliably separate unknown objects from the background. Once an object has been identified, the robot can continuously manipulate it to accumulate more information about it and learn its complete visual appearance. We demonstrate the feasibility of the proposed approach by applying it to the problem of autonomous learning of visual representations for viewpoint-independent object recognition on a humanoid robot.

Keywords

Humanoid robotics developmental robotics object perception active vision

Get full access to this article

View all access options for this article.

References

Asfour

Regenstein

Azad

Schröder

Bierbaum

Vahrenkamp

Dillmann

(2006). ARMAR-III: An integrated humanoid platform for sensory-motor control. In 2006 6th IEEE-RAS Int. Conf. Humanoid Robots (p. 169–175). Genoa, Italy.

Beder

Förstner

(2006). Direct solutions for computing cylinders from minimal sets of 3D points. In European Conf. Computer Vision (ECCV) (p. 135–146).

Browatzki

Tikhanoff

Metta

Bülthoff

H. H.

Wallraven

(2012). Active object recognition on a humanoid robot. In IEEE Int. Conf. Robotics and Automation (ICRA) (p. 2021–2028). Saint Paul, Minnesota.

Chang

Smith

J. R.

Fox

(2012). Interactive singulation of objects from a pile. In IEEE Int. Conf. Robotics and Automation (ICRA) (p. 3875–3882). Saint Paul, Minnesota.

Chaperon

Goulette

(2001). Extracting cylinders in full 3D data using a random sampling method and the Gaussian image. In Proc. Vision Modeling and Visualization Conference.

Cheng

Hyon

S.-H.

Morimoto

Ude

Hale

J. G.

Colvin

Scroggin

Jacobsen

S. C.

(2007). CB: A humanoid research platform for exploring neuroscience. Advanced Robotics, 21(10), 1097–1114.

Crammer

Singer

(2001). On the algorithmic implementation of multiclass kernel-based vector machines. Journal of Machine Learning Research, 2, 265–292.

Csurka

Dance

Fan

L. X.

Willamowski

Bray

(2004). Visual categorization with bags of keypoints. In Proc. ECCV Int. Workshop Statistical Learning in Computer Vision. Prague, Czech Republic.

Fei-Fei

Fergus

Perona

(2006). One-shot learning of object categories. IEEE Transactions on Pattern Recognition and Machine Intelligence, 28(4), 594–611.

10.

Feldman

(2003). What is a visual object? Trends in Cognitive Sciences, 7(6), 252–256.

11.

Fischler

M. A.

Bolles

R. C.

(1981). Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381–395.

12.

Fitzpatrick

Metta

(2003). Grounding vision through experimental manipulation. Philosophical Transactions of the Royal Society – Series A, 361(1811), 2165–2185.

13.

Fitzpatrick

Needham

Natale

Metta

(2008). Shared challenges in object perception for robots and infants. Infant and Child Development, 17, 7–24.

14.

Foissotte

Stasse

Wieber

P.-B.

Escande

Kheddar

(2010). Autonomous 3d object modeling by a humanoid using an optimization-driven next-best-view formulation. International Journal of Humanoid Robotics, 7(3), 407–428.

15.

Forssen

(2007). Maximally stable colour regions for recognition and matching. Minneapolis, Minnesota.

16.

Gibson

(1979). The ecological approach to visual perception. Boston, MA: Houghton Mifflin.

17.

Gupta

Sukhatme

G. S.

(2012). Using manipulation primitives for brick sorting in clutter. In IEEE Int. Conf. Robotics and Automation (ICRA) (p. 3883–3889). Saint Paul, Minnesota.

18.

Harris

Stephens

(1988). A combined corner and edge detector.

19.

Horn

B. K. P.

(1987). Closed-form solution of absolute orientation using unit quaternions. Journal Optical Society America A, 4(4), 629–642.

20.

Johnson

S. P.

Aslin

R. N.

(1996). Perception of object unity in young infants: The roles of motion, depth, and orientation. Cognitive Development, 11, 161–180.

21.

Jüttner

Müller

Rentschler

(2006). A developmental dissociation of view-dependent and view-invariant object recognition in adolescence. Behavioural Brain Research, 175(2), 420–424.

22.

Katz

Brock

(2008). Manipulating articulated objects with interactive perception. In IEEE Int. Conf. Robotics and Automation (ICRA) (p. 272–277). Pasadena, CA.

23.

Kellman

P. J.

(1993). Kinematic foundations of infant visual perception. In Granrud

C. E.

(Ed.), Visual perception and cognition in infancy (p. 121–173). Hillsdale, NJ: Erlbaum.

24.

Kenney

Buckley

Brock

(2009). Interactive segmentation for manipulation in unstructured environments. In IEEE Int. Conf. Robotics and Automation (ICRA) (1377–1382). Kobe, Japan.

25.

Kraebel

K. S.

West

R. N.

Gerhardstein

(2007). The influence of training views on infant’s long-term memory for simple 3D shapes. Developmental Psychobiology, 49(4), 406–420.

26.

Kraft

Pugeault

Baseski

Popovic

Kragic

Kalkan

Wörgötter

Krüger

(2008). Birth of the object: Detection of objectness and extraction of object shape through object-action complexes. International Journal of Humanoid Robotics, 5(2), 247–265.

27.

Krainin

Henry

Ren

Fox

(2011). Manipulator and object tracking for in-hand 3D object modeling. The International Journal of Robotics Research, 30(11), 1311–1327.

28.

W. H.

Kleeman

(2011). Segmentation and modeling of visually symmetric objects by robot actions. The International Journal of Robotics Research, 30(9), 1124–1142.

29.

Lowe

D. G.

(1999). Object recognition from local scale-invariant features. In Int. Conf. Computer Vision (ICCV) (1150–1157). Corfu, Greece.

30.

Matas

Chum

Urba

Pajdla

(2004). Robust wide-baseline stereo from maximally stable extremal regions. Image and Vision Computing, 22(10), 761–767.

31.

Metta

Fitzpatrick

(2003). Better vision through manipulation. Adaptive Behavior, 11(2), 109–128.

32.

Moreels

Perona

(2007). Evaluation of features detectors and descriptors based on 3D objects. International Journal of Computer Vision, 73, 263–284.

33.

Nishimura

Scherf

Behrmann

(2009). Development of object recognition in humans. F1000 Biology Reports, 1(56).

34.

Palmer

Rock

(1994). Rethinking perceptual organization: the role of uniform connectedness. Psychonomic Bulletin and Review, 1, 29–55.

35.

Peissig

J. J.

Tarr

M. J.

(2007). Visual object recognition: Do we know more now than we did 20 years ago? Annual Review of Psychology, 58, 75–96.

36.

Pelleg

Moore

(2000). X-means: Extending k-means with efficient estimation of the number of clusters. In Proc. 17th Int. Conf. Machine Learning (p. 727–734). San Francisco, CA.

37.

Ramisa

Vasudevan

Scaramuzza

de Mántaras

R. L.

Siegwart

(2008). A tale of two object recognition methods for mobile robots. In Int. Conf. Computer Vision Systems (ICCV) (p. 353–362). Santorini, Greece.

38.

Roth

Ommer

(2006). Exploiting low-level image segmentation for object recognition. In Pattern Recognition (p. 11–20). Berlin, Heidelberg: Springer-Verlag.

39.

Schaal

Peters

Nakanishi

Ijspeert

(2005). Learning movement primitives. In Robotics Research: The Eleventh International Symposium (p. 561–572). Berlin, Heidelberg: Springer-Verlag.

40.

Schiebener

Ude

Morimoto

Asfour

Dillmann

(2011). Segmentation and learning of unknown objects through physical interaction. In 2011 11th IEEE-RAS Int. Conf. Humanoid Robots (p. 500–506). Bled, Slovenia.

41.

Shibata

Vijayakumar

Conradt

Schaal

(2001). Biomimetic oculomotor control. Adaptive Behavior, 9(3–4), 189–207.

42.

Smith

W. C.

Johnson

S. P.

Spelke

E. S.

(2003). Motion and edge sensitivity in perception of object unity. Cognitive Psychology, 46, 31–46.

43.

Spelke

E. S.

(1990). Principles of object perception. Cognitive Science, 14, 29–56.

44.

Stergaršek Kuzmič

Ude

(2010). Object segmentation and learning through feature grouping and manipulation. In 2010 10th IEEE-RAS Int. Conf. Humanoid Robots (p. 371–378). Nashville, Tennessee.

45.

Tsikos

C. J.

Bajcsy

R. K.

(1991). Segmentation via manipulation. IEEE Transactions on Robotics and Automation, 7(3), 306–319.

46.

Ude

Omrčen

Cheng

(2008). Making object learning and recognition an active process. International Journal of Humanoid Robotics, 5(2), 267–286.

47.

Ude

Oztop

(2009). Active 3-D vision on a humanoid head. In International Conference on Advanced Robotics (ICAR). Munich, Germany.

48.

Vahrenkamp

Berenson

Asfour

Kuffner

Dillmann

(2009). Humanoid motion planning for dual-arm manipulation and re-grasping tasks. In IEEE/RSJ Int. Conf. Intelligent Robots and Systems (IROS) (p. 2464–2470). St. Louis, MO.

49.

Welke

Issac

Schiebener

Asfour

Dillmann

(2010). Autonomous acquisition of visual multi-view object representations for object recognition on a humanoid robot. In IEEE Int. Conf. Robotics and Automation (ICRA) (p. 2012–2019). Anchorage, Alaska.