In this work, we propose to reconstruct a complete three-dimensional (3-D) model of an unknown object by fusion of visual and tactile information while the object is grasped. Assuming the object is symmetric, a first hypothesis of its complete 3-D shape is generated. A grasp is executed on the object with a robotic manipulator equipped with tactile sensors. Given the detected contacts between the fingers and the object, the initial full object model including the symmetry parameters can be refined. This refined model will then allow the planning of more complex manipulation tasks. The main contribution of this work is an optimal estimation approach for the fusion of visual and tactile data applying the constraint of object symmetry. The fusion is formulated as a state estimation problem and solved with an iterated extended Kalman filter. The approach is validated experimentally using both artificial and real data from two different robotic platforms.
AllenP (1984) Surface descriptions from vision and touch. In: IEEE international conference on robotics and automation, pp. 394–397.
2.
AllenP (1988) Integrating vision and touch for object recognition tasks. The International Journal of Robotics Research7(6): 15–33.
3.
AstolaJHaavistoPNeuvoY (1990) Vector median filters. Proceedings of the IEEE78(4): 678–689.
4.
BalasubramanianRXuLBrookPD. (2012) Physical human interactive guidance: Identifying grasping principles from human-planned grasps. IEEE Transactions on Robotics28(4): 899–910.
5.
BerensonDSrinivasaSKuffnerJ (2011) Task space regions: A framework for pose-constrained manipulation planning. The International Journal of Robotics Research30(12): 1435–1460.
6.
BicchiAKumarV (2000) Robotic grasping and contact. In: IEEE international conference on robotics and automation, San Francisco, CA.
7.
BierbaumARambowMAsfourT. (2008) A potential field approach to dexterous tactile exploration of unknown objects. In: IEEE-RAS international conference on humanoid robots, pp. 360–366.
8.
BjörkmanMKragicD (2010) Active 3D scene segmentation and detection of unknown objects. In: IEEE international conference on robotics and automation, pp. 3114–3120.
9.
BohgJJohnson-RobersonMLeonB. (2011) Mind the gap – Robotic grasping under incomplete observation. In: IEEE international conference on robotics and automation, pp. 686–693.
10.
ColomboCDel BimboAPerniciF (2005) Metric 3D reconstruction and texture acquisition of surfaces of revolution from a single uncalibrated view. IEEE Transactions on Pattern Analysis and Machine Intelligence27(1): 99–114.
11.
DillmannR (2007) KIT ObjectModels web database. Institute of Anthropomatics, Humanoids and Intelligence Systems Laboratories, Karlsruhe Institute of Technology. Available at: http://i61p109.ira.uka.de/ObjectModelsWebUI.
12.
DragievSToussaintMGiengerM (2011) Gaussian process implicit surfaces for shape estimation and grasping. In: IEEE international conference on robotics and automation, pp. 2845–2850.
13.
FerrariCCannyJ (1992) Planning optimal grasps. In: IEEE international conference on robotics and automation, pp. 2290–2295.
14.
HaidacherSHirzingerG (2003) Estimating finger contact location and object pose from contact measurements in 3D grasping. In: IEEE international conference on robotics and automation, pp. 1805–1810.
15.
HartleyRZissermanA (2003) Multiple View Geometry in Computer Vision. 2nd edn.Cambridge, UK: Cambridge University Press.
16.
HebertPHudsonNMaJ. (2011) Fusion of stereo vision, force-torque, and joint sensors for estimation of in-hand object location. In: IEEE international conference on robotics and automation, pp. 5935–5941.
17.
HenryPKraininMHerbstE. (2012) RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments. The International Journal of Robotics Research31(5): 647–663.
18.
HsiaoKChittaSCiocarlieM. (2010) Contact-reactive grasping of objects with partial shape information. In: IEEE international conference on intelligent robots and systems, pp. 1228–1235.
19.
IlonenJKyrkiV (2011) Robust robot-camera calibra- tion. In: International conference on advanced robotics, pp. 67–74.
20.
JiangYMosesonSSaxenaA (2011) Efficient grasping from RGBD images: Learning using a new rectangle representation. In: IEEE international conference on robotics and automation, pp. 3304–3311.
21.
KraininMHenryPRenX. (2011) Manipulator and object tracking for in-hand 3D object modeling. The International Journal of Robotics Research30(11): 1311–1327.
22.
LaiKBoLRenX. (2011) Sparse distance learning for object recognition combining RGB and depth information. In: IEEE international conference on robotics and automation.
23.
LeeDHebertMKanadeT (2009) Geometric reasoning for single image structure recovery. In: IEEE conference on computer vision and pattern recognition, pp. 2136–2143.
24.
MartonZCPangercicDBlodowN. (2011) Combined 2D–3D categorization and classification for multimodal perception systems. The International Journal of Robotics Research30(11): 1378–1402.
25.
MeierMSchopferMHaschkeR. (2011) A probabilistic approach to tactile shape reconstruction. IEEE Transactions on Robotics27(3): 630–635.
26.
NatarajanSRistic-DurrantDLeuA. (2011) Robust stereo-vision based 3D modelling of real-world objects for assistive robotic applications. In: IEEE international conference on intelligent robots and systems, pp. 786–792.
27.
NguyenVD (1989) Constructing stable grasps. The International Journal of Robotics Research8(1): 26–37.
28.
PopovicMKootstraGJorgensenJA. (2011) Grasping unknown objects using an early cognitive vision system for general scene understanding. In: IEEE international conference on intelligent robots and systems.
29.
RoyMFoufouSTruchetetF (2004) Mesh comparison using attribute deviation metric. International Journal of Image and Graphics4:
30.
RusuRBBradskiGThibauxR. (2010) Fast 3D recognition and pose using the viewpoint feature histogram. In: IEEE international conference on intelligent robots and systems.
31.
SaxenaADriemeyerJNgAY (2008) Robotic grasping of novel objects using vision. The International Journal of Robotics Research27(2): 157–173.
32.
SimonD (2006) Optimal State Estimation. New York, NY: John Wiley & Sons.
33.
TaubinG (1991) Estimation of planar curves, surfaces, and nonplanar space curves defined by implicit equations with applications to edge and range image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence13(11): 1115–1138.
34.
ThrunSWegbreitB (2005) Shape from symmetry. In: IEEE international conference on computer vision, pp. 1824–1831.
35.
WohlkingerWVinczeM (2011) Shape-based depth image to 3D model matching and classification with inter-view similarity. In: IEEE international conference on intelligent robots and systems, pp. 4865–4870.
36.
WuthrichMPastorPRighettiL. (2012) Probabilistic depth image registration incorporating nonvisual information. In: IEEE international conference on robotics and automation, pp. 3637–3644.
37.
XueTLiuJTangX (2011) Symmetric piecewise planar object reconstruction from a single image. In: IEEE conference on computer vision and pattern recognition, pp. 2577–2584.
38.
YamadaYIshiguroAUchikawaY (1993) A method of 3D object reconstruction by fusing vision with touch using internal models with global and local deformations. In: IEEE international conference on robotics and automation, pp. 782–787.
