This paper presents a robotic perceptual system which utilizes passive vision and active touch. The task is one-fingered exploration of a single unmodeled object for apprehension— the determination of the features of the object and the rela tions among them. A two-stage exploration is utilized. Vision is first used in a feedforward manner to segment the object and to obtain its position. Touch is then used in a feedback mode to further explore the object. In designing this system, we have addressed several issues. The first concerns the way in which the robotic perceptual system should be structured. The model which we propose here is based upon theories of human perception. It consists of a highly modularized set of knowledge-based modules, each of which is domain specific and informationally encapsulated. Within the framework of this model, we have designed both vision and touch subsys tems. In each case, we have defined the primitives, features, and representations extracted and created by the system. The visual system is passive and relatively simple. The touch system is active and is an attempt to systematically structure a robotic haptic perception system. Finally, we have ad dressed the issues of how these two subsystems interact dur ing active exploration and how the information from each is integrated. The work described has been implemented and tested on a robot system consisting of a combination tactile array/force-torque sensor, a PUMA robot arm, a pair of CCD cameras, and a Vax 750.
Get full access to this article
View all access options for this article.
References
1.
Allen, P.1985. Object recognition using vision and touch. Ph.D. thesis, CIS Dept., University of Pennsylvania, Philadelphia, Pa.
2.
Arbib, M., Iberal, T., and Lyons, D.1983. Coordinated control programs for movements of the hand . Technical Report COINS 83-25, CIS Dept., UMass Amherst , Amherst, Mass.
3.
Bajcsy, R., Brown, D., Wolfeld, J., and Peters, D.1982. What can we learn from one finger experiments? A Report on a Joint US-France NSF-CNRS Workshop on Advanced Automation and Robotics , pp. 119-158.
4.
Bernstein, H.1984. Determining the shape of a convex n-sided polygon by using 2n + k tactile probes. Technical Report 125, NYU, Courant Institute, New York.
5.
Browse, R., and Lederman, S.1985. A framework for robotic perception. Technical Report 85-165, Queen's University, Kingston , Canada.
6.
Buchner, S.1985. Installation and operations manual for LTS-200 tactile sensor. Raleigh, N.C.: LORD Corp.
7.
Crowley, J.1986 (San Francisco, Calif.). Representation and maintenance of a composite surface model. Proc. IEEE Conf. on Robotics and Automation, pp. 1455-1461.
8.
Dario, P., Fiorillo, A., Buttazzo, G., and Francesconi, R.1985 (Tokyo). A sensorized scenario for basic investigation on active touch. Proc. Int. Conf. on Advanced Robotics.
9.
Durrant-Whyte, H.1986. Integration, coordination and control of multisensor robot systems. Ph.D. thesis, CIS Dept., University of Pennsylvania , Philadelphia, Pa.
10.
Ellis, R.1984. Extraction of tactile features by passive and active sensing. Intelligent Robots and ComputerVision521:289-299.
11.
Fearing, R., and Hollerbach, J.1985. Basic solid mechanics for tactile sensing. Int. J. Robotics Res.4(3):40-54.
12.
Fodor, J.1983. The modularity of mind. Cambridge, Ma.: MIT Press.
13.
Fuma, F., Krotkov, E. and Summers, J.1986. The Pennsylvania active camera system. Technical Report MS-CIS-86-15, CIS Dept., University of Pennsylvania, Philadelphia, Pa.
14.
Grimson, W.1981. From images to surfaces: A computational study of the human early visual system. Cambridge, Mass.: MIT Press.
15.
Grimson, W., and Lozano-Perez, T.1983. Model-based recognition and localization from sparse range or tactile data. Technical Report 738, Artificial Intelligence Laboratory, MIT, Cambridge, Mass.
16.
Harmon, L.1982. Automated tactile sensing. Int. J. Robotics Res.1(2):3-32.
17.
Harmon, S., Bianchini, G., and Pinz, B.1986 (San Francisco, Calif.). Sensor data fusion through a distributed blackboard. Proc. IEEE Conf. on Robotics and Automation, pp. 1449-1454.
18.
Hillis, W.1982. A high-resolution imaging touch sensor. Int. J. Robotics Res.2(3):33-44.
19.
Horn, B.1975. Shape from shading. In Psychology of computer vision, ed. P. Winston.New York: McGraw-Hill.
20.
Jeannerod, M., and Biguer, B.1982. Visuomotor mechanisms in reaching within extrapersonal space. In Analysis of visual behavior, eds. D. Ingle, M. Goodale, and R. Mansfield.Cambridge, Mass.: MIT Press, pp. 387-409.
21.
Kender, J.1978 (Nov.). Shape from texture: a brief overview and a new aggregation transform. Proc. DARPA Image Understanding Workshop, pp. 79-84.
22.
Kinoshita, G., Aida, S., and Mori, M.1975. A pattern classification by dynamic tactile sense information processing. Pattern Recognition7:243-250.
23.
Klatzky, R., Lederman, S., and Metzger, V.1985. Identifying objects by touch: an expert system. Perception and Psychophysics37(4):299-302.
24.
Klatzky, R., McCloskey, B., Doherty, S., Pellegrino, J., and Smith, T.1986. Hand shaping and object processing. Technical Report Cognitive Science 8605, University of California at Santa Barbara.
25.
Krotkov, E.1987. Exploratory visual sensing for determining spatial layout with an agile stereo camera system. Ph.D. thesis, CIS Dept., University of Pennsylvania, Philadelphia, Pa.
26.
Lederman, S., and Klatzky, R.1985 (Hennicker, N.H.). Knowledge-based control of human hand movements. Proc. Conf. on Biomechanics and Neural Control of Movement.
27.
Lederman, S., and Klatzky, R.1986. Hand movements: a window into haptic object recognition . Technical Report Cognitive Science 8606, University of California at Santa Barbara.
28.
Luo, R., and Tsai, W.1986 (San Francisco, Cavif.). Object recognition using tactile image array sensors. Proc. IEEE Conf. on Robotics and Automation, pp. 1248-1253.
29.
Marr, D.1982. Vision. San Francisco: W. H. Freeman.
30.
Overton, K.1984. The acquisition, processing, and use of tactile sensor data in robot control. Technical Report, UMass Amherst .
31.
Ozaki, H., Waku, S., Mohri, A., and Takata, M.1982. Pattern recognition of a grasped object by unit vector distribution. IEEE Trans. Systems, Man, Cybernetics SMC-12(3):315-324.
32.
Paillard, J.1982. The contribution of peripheral and central vision to visually guided reaching. In Analysis of visual behavior, eds. D. Ingle, M. Goodale, and R. Mansfield.Cambridge, Mass: MIT Press, pp. 367-385.
33.
Peterson, R.1984 (Bethlehem, Pa.). A compliant tactile sensor utilizing conductive elastomers. Proc. First World Conf. on Robotics Research.
34.
Rebman, J., and Morris, K.1983 (Cambridge, Mass.). A tactile sensor with electrooptical transduction. Proc. 3rd Int. Conf. on Robot Vision and Sensory Controls, pp. 341-347.
35.
Salisbury, J.1984 (Atlanta, Ga.). Interpretation of contact geometries from force measurements. Proc. IEEE Conf. on Robotics and Automation, pp. 240-247.
36.
Shafer, S., Stenz, A., and Thorpe, C.1986 (San Francisco, Calif.). An architecture for sensor fusion in a mobile robot. Proc. IEEE Conf. on Robotics and Automation, pp. 2002-2011.
37.
Shen, J., Loparo, K., and Thomas, C.1986 (Detroit, Mich.). An active touch processing system. Proc. Sensors 86 Conf
38.
Smitley, D.1985. The design and analysis of a stereo vision algorithm. Master's thesis, CIS Dept., University of Pennsylvania , Philadelphia, Pa.
39.
Stansfield, S.Angy: a rule-based expert system for automatic segmentation of coronary vessels from digital subtracted angiograms. IEEE Trans. Pattern Analysis Machine Intelligence PAMI-8(2):188-199.
40.
Stansfield, S.1987. Visually-guided haptic object recognition. Ph.D. thesis, CIS Dept., University of Pennsylvania, Philadelphia, Pa.
41.
Talton, D.1985. Implementation of a Gaussian-smoothing grandient-based edge detector. Technical Report GRASPLab TR 35, CIS Dept., University of Pennsylvania, Philadelphia, Pa.
42.
Terzopolous, D.1982. Multi-level reconstruction of surfaces. Technical Report 671, AI Lab, MIT, Cambridge, Mass.
