Abstract
In this paper, a new object recognition algorithm employing a curvature-based histogram is presented. Recognition of three-dimensional (3-D) objects using range images remains one of the most challenging problems in 3-D computer vision due to its noisy and cluttered scene characteristics. The key breakthroughs for this problem mainly lie in defining unique features that distinguish the similarity among various 3-D objects. In our approach, an object detection scheme is developed to identify targets underlining an automated search in the range images using an initial process of object segmentation to subdivide all possible objects in the scenes and then applying a process of object recognition based on geometric constraints and a curvature-based histogram for object recognition. The developed method has been verified through experimental tests for its feasibility confirmation.
1. Introduction
Recently developed computer vision technology has an emerging interest in range images for important automation applications such as human surveillance [1, 2], car crash prevention [3], autonomous robot navigation [4], military target recognition [5], etc. The main task of traffic monitoring applications is to track and recognize moving targets such as cars, trucks, motorbikes and people. The core problems of feature extraction of objects from range images can appear in many aspects with a focus on data quality and the nature of object shapes. In some applications, the feature extraction process is performed solely on the basis of range images, while in others some prior information about the object shape's features or characteristics are needed in advance. With the availability of range sensing devices that can provide real-time and high-resolution 3-D data [6, 7], research on analysing shapes and investigating similarity relations between the geometric and visual shape of object for recognition purposes has been performed extensively in the fields of computer vision, artificial intelligence and computer-aided design [8, 9].
The first important task for object recognition lies in object segmentation. The main issue in object segmentation is to precisely cluster scanned point clouds into foreground and background [10, 11, 12]. Classified by segmentation, the points belonging to the same object are grouped and marked, which is helpful for further steps in object recognition and 3-D feature extraction.
Apart from object segmentation, object recognition using range images contains a number of fundamental problems, such as identification of the object and estimation of its location and viewing angle and sometimes multiple views are required for fully retrieving the correct matched model in the library. The term “object recognition” has been used in computer vision as an important keyword, which describes the entire process of automatic identification and estimation of objects. Object recognition includes the important process of determining the similarity between the inspecting object and the model stored in a database. This is often performed by computing the distance between feature vectors. In general, the recognition has to search among the possible candidate features for identification of the best match and then assign the label to the matched object in the scene. Based on the characteristics of the shape descriptor, the object recognition methods can be divided into two main categories: global feature methods and local feature methods.
Generally, global feature techniques can characterize the global shape features of a 3-D model, such as volume, area and Fourier transform coefficients [13], bounding boxes, moments-based and wavelet-based descriptors [14], hull packing and hull compactness [15], moments-based classifier and weighted Euclidean distance measure [16], histograms of diffusion distance, geodesic distance, the ratio of diffusion and geodesic distances and a curvature weighted distance [17]. One of the important techniques in this category was introduced in [18] by comparing the object shape distributions and measuring the geometric properties based on distance, angle, area and volume measurements between random surface points. The implementation of these methods is relatively straightforward with an easily defined distance for comparison in the matching process. Since the global features are not especially discriminative about object details or undesirably sensitive to optical occlusion, these methods are still yet to be further applied to practical applications. On the other hand, these methods may be employed as an active pre-filter, or they can be used in combination with other methods to improve performance.
The other methods are based on local feature characteristics. Local feature extraction for object recognition has been one of the key research issues being addressed. Local features can be extracted from interest points or local regions, such as the surface curvature mapped in a spherical coordinate system of 3-D objects [19], point signatures that accumulate surface information along a 3-D curve in the neighbourhood of a point [20], tensor representation with a 3-D descriptor of area intersection around a point [21, 22], 3-D shape contexts as semi-local descriptions of object shapes, centred at points on the surface of the object [23, 24], local surface patch (LSP) defined as distribution of shape index and the angle between the inspecting point and its neighbours [25]. One effective and newly developed technique using local features is two-dimensional (2-D) scale invariant feature transform (SIFT) [26, 27, 28]. In this technique, the locations of some key points are first identified on the object surface, where the shape indexes are the maxima. A 2-D histogram is formed around every key point for object matching. The two dimensions of the 2-D histogram normally include two index definitions: one is the shape index values of the neighbouring pixels and another is the angles between the normal of the key point and the normal of the neighbouring pixels. These studies only used isolated objects, in which the problematic issues in boundary regions can be easily eliminated by simple thresholding. However, these methods may still face a severe problem in dealing with a complicated condition further encountered by issues such as surface clutter or occlusion, which haven't been considered and evaluated in these studies.
Spin image [29, 30, 31] is another object matching technique, which uses 2-D image characterization of a point belonging to a surface. The 2-D image shape descriptor has two coordinates of alpha and beta, which are calculated for a vertex in the surface mesh. Another two important parameters of this technique are the supporting length and bin size. The supporting length is an index, which sets the maximum distance between the oriented point and a neighbouring point contributing to the spin image and the bin size decides the resolution of the 2-D image shape descriptor. Since the spin image counts the number of points within the supporting length and divides them to the equal and fixed bin size regions, the algorithm is undesirably sensitive to variation in data resolution.
Up to now, most developed methods are either time-consuming or have problems with memory overloading and may lack the capability for handling real-time object feature extraction or recognition. Therefore, in this research, we aim to make progress by developing a curvature-based histogram technique, in which matching needs much less computation time and memory in object recognition.
The article is structured as follows. The object segmentation method is introduced in the next section with a specific description of a region growing algorithm using surface characteristics for locating the objects. The main proposed method for object recognition employing curvature-based histogram is introduced in Section 3. Section 4 describes the experimental design and analyses some recognition results from detecting some real scenes. Section 5 concludes the paper.
2. Object Segmentation
To obtain the segmented point cloud of an object for later object recognition, an initial object segmentation process is applied to the acquired range image. Details of the method for object segmentation are described in [2]. By classifying point clouds in range images into three different surface types, namely uniform, rough and noisy classes, based on the distribution of the normal surface vector, the scene ground is detected by applying least-squares plane fitting to the surface region being classified as the uniform class. Furthermore, to localize the scene ground accurately, a region growing process is developed to extend the homogeneous region of an object region initially segmented by the first stage. As a result, all the individual objects existing in range images can be segmented by using a recursive search algorithm after the scene ground is removed. A series of steps for implementation of the proposed method is shown in Figure 1.

Flow chart of the object segmentation method.
After the segmentation process, all the possible objects in the range image are marked for the next step of object recognition. In some other research [11, 16, 17, 19], the object segmentation process is omitted and the object recognition process is applied directly to the range image by exhaustive searching for all the points in the image to find the corresponding points with the object models. Those methods may be time consuming due to the fact that the recognition process is especially burdensome since the objects to be matched may only occupy rather small space regions in the overall searching space. In this research, the object recognition is divided into two stages for fast and efficient object segmentation to be performed before the object recognition process is implemented. Examples of some segmented point clouds obtained by object segmentation are shown in Figure 2.

Segmentation examples of the point clouds: (a) street segmentation, (b) three object categories obtained from segmentation process: human being, cars and motorbikes.
3. Object recognition
We define a framework for object recognition using a combination of various object features. The proposed method may also be called a hybrid technique since it uses a global-based feature extracted from the histogram of the surface curvature for automatic localization and matching with the respectful object model in specific viewing angles. By describing an object as a set of 1-D histograms of the surface curvature that is invariant even when extracted from different resolutions of an object model, the proposed method is capable of recognizing objects suffering severe noises. The object recognition goal is achieved by finding the correlation of relevant dimensions and a curvature-based histogram between the segmented point clouds and the object model stored in the database, shown in Figure 3. The features of the segmented point clouds, including the critical object dimension and curvature-based histograms obtained by the object segmentation process, are calculated. The developed algorithm for object recognition includes two processes:

Object recognition process by searching similarity features.

Object orientation determination by calculating three eigenvectors of covariance matrix.
Establish the object models in a database (offline process): Dimension and curvature-based histograms of 3-D object models are constructed offline and stored in the database.
Match acquired range data with the object models (online process): During the online process of object recognition, dimension and curvature-based histograms of segmented object are calculated and compared with the object models in database to find the correct match.
3.1 Object dimension
Three parameters of dimension including
To calculate the three parameters,
where
Eq. (1) can be rewritten as:
Since 5 is a symmetric matrix, some conclusions can be made:
Eigenvalues are all real.
Eigenvectors are orthogonal.
In this research, the GNU Scientific Library is used to calculate the eigenvalues of 5, denoted as
For comparing the three parameters (

Membership functions of the fuzzy set for finding correlation value of three defined parameters (
where
3.2 Curvature-based histogram
In the proposed method, a histogram is defined and employed as a feature for object matching. We denote
where
The
This function results in the range (–
Then the curvature-based histogram of a segmented point cloud, S, is given by:
where
The proposed method employs this curvature-based histogram as a feature for object matching. This angle ranges from 0 to 360 degrees due to the use of the

Curvature-based histogram at a viewpoint for the car model: (a) Different views of
The calculation of similarity between
The correlation coefficient
Due to the fact that acquired data from a range image is actually only one view of the object, the proposed method uses a set of histograms with different viewing angles for searching for the right object for matching. We use the artificial viewpoint, which is placed in different locations with different yaw angles to generate histograms from the model in database, shown in Figure 7. The yaw angle of the object coordinate is taken from the orientation calculation process. A total of 360° degrees of yaw angle is divided by an equal pace of

Setting of a viewpoint for a model in database to generate a histogram.
When comparing a segmented point cloud with a model in the database, the histogram of the point cloud is compared with all

Flow chart of the searching process among all the viewing angles.
The histogram-matching coefficient is selected as the highest correlation coefficient among all the viewing angles:
3.3 Matching Coefficient
The final matching coefficient between a segmented point cloud and a model in database is calculated as follows:
where
The weighting coefficients,
4. Experimental results and analysis
4.1 Laser scanner system setup
The proposed method has been tested using more than 50 range images obtained by a single-point laser scanner (RIEGL 90-3800), shown in Figure 9. To perform full-field scanning, two servomotors were used for the acquisition of whole-view point clouds. One rotation motor was employed to control the vertical scanning pitch, which was synchronized with RIEGL 90-3800. The laser clock of this device was turned on when the vertical motor was run at a constant speed for acquiring range data. After reading one vertical line, the other motor that continuously controlled the horizontal scanning pitch shifted a scanning pitch. Data were acquired simultaneously and fed to the computer during line scanning. The 2D scan resolution was set at 300×300 pixels and the scanning time was about 90 seconds on average. The range accuracy of RIEGL 90-3800 is 0.05m in an overall measuring range of 10 to 1000m.

Laser scanner system employed in the experiment.
4.2 Feasibility of the proposed method
To generate point clouds of object models in the database, we used the Princeton Shape Benchmark containing a database of 3-D polygonal models (1,814 models in total, available at http://shape.cs.princeton.edu/benchmark/), in which the 3-D model is stored in an Object File Format (OFF) file with the polygonal geometry. We generated uniform sampling point clouds for each object model from the OFF file using our simulation software. We also generated point clouds of object models by converting CAD models using the same software. The proposed method has been tested by an extensive image set. The number (
In the first experiment, the method was first tested with object models in the database for object searching and recognizing. Five car models, including a Ford Escape, a Honda Civic, a Honda CRV, a Mitsubishi Savrin and a Toyota Camry, were used for finding the correct matching model in the database by the proposed method. For the rest of the experiment conducted in this paper, all of the eight car models were used in the matching process. At first, the viewing angles of the five cars were randomly generated and their curvature-based histograms were calculated for matching with the histograms of other car models in the database, as shown in Table 1. The matching results showed that the proposed method could successfully retrieve the correct model of the inquired car. Due to the viewing angle pace, (
Matching results of curvature-based histograms for five car models: Ford Escape, Honda Civic, Honda CRV, Mitsubishi Savrin and Toyota Camry.
The method was tested with a real range image in which a car (Honda-CRV) was parking behind a post and a tree. The curvature-based histograms of the Honda-CRV model were generated for matching in the database, shown in Figure 10. As seen in Figure 11 and Table 2, the car was accurately segmented and recognized automatically with an average matching coefficient for the car set of 0.806 and the model having the highest matching coefficient,

Generating curvature-based histograms for a car model (Honda-CRV).

Searching result of a Honda-CRV car (matching coefficient
Recognition results of segmented objects when comparing to the car set
In another experiment, two models were selected for the object detection process: the Ford-Escape and the Honda-Civic. As seen in Figure 12, using the proposed method, all the cars were retrieved accurately. The highest matching coefficients of the two cars belonging to their correct models, the Ford-Escape and the Honda-Civic, are 0.918 and 0.935, respectively.

Searching result for Ford-Escape (matching coefficient
The car recognition process using the proposed method was performed in several scenarios, shown in Figure 13. With difference target arrangements, the examples with optical occlusion and randomness of car orientation may encounter some minor difficulties in car recognition. However, the method still worked effectively in most cases. For all the images tested, the successful rate of object recognition is 90.4%. The time measurement of the object recognition task is summarized in Figure 14. The average time for processing one range image is calculated at about 4.2 seconds when a 2.4GHz of CPU equipped with 3GB RAM is employed. The technique can yield a considerable improvement in both the speed and accuracy of object recognition. The running time of the proposed recognition algorithm gives promising results for real-time applications since the traditional techniques such as spin image and local surface patch required several minutes to hours for implementing the recognition task of one range image.

Recognition results for cars in range images at different scenarios.

Time consumption for object recognition by proposed method.
5. Conclusions
In this study, a new method for automatic object detection has been developed for general laser scanner systems acquiring range images. The proposed method includes an initial object segmentation process and an object recognition algorithm using geometrical constraints and a curvature-based histogram. The method can effectively recognize objects in range images with a success rate of 90.4% and the time consumption for processing each range image of 300×300 pixels is 4.2 seconds on average when using a conventional personal computer. The histogram is invariant when extracted from different resolutions of an object model. The robustness of the method can be further enhanced when the issues in dealing with 3-D images with a poor S/N ratio and severe optical occlusion are improved. Future research will concentrate on increasing the speed of the object recognition process and parallel programming will be applied, in which program instructions will be divided among multiple processors for acquiring, processing and displaying range images. In regard to that, three- or four-core CPUS might be taken into consideration for running the proposed algorithm.
