Abstract
To achieve accurate visual object tracking and overcome the difficulties brought by the object deformation, occlusion, and illumination variations, a particle filter for object tracking algorithm based on color local entropy (CLE) is proposed. First we improved the traditional histogram weighted function by using a scale factor. Then, for the shortcoming that the color feature is sensitive to illumination and environmental interference, a color local entropy object observation model is constructed by mapping the object from color feature space to local entropy space. In addition, an adaptive updating strategy of the object template is designed and the number of particle is adjusted dynamically according to the tracking performance. The experimental results show that compared with several existing algorithms, the proposed algorithm is more effective and robust for the real-time object tracking under the situation of illumination variation, object occlusion, and nonlinear motion.
1. Introduction
The real-time object tracking as the key step of Intelligent Transportation System (ITS) has been paid more and more attention. It is the premise of the traffic behavior analysis and prediction. Because of the complexity of the traffic environment, tracking moving object within a video sequence is still a challenging task. The difficulties of object tracking are mainly including the variation of object itself, such as pose variation in the process of the object motion and nonrigid object shape deformation; the extrinsic environment interference, such as illumination variation, occlusion, and camera vibration; and the real-time restriction.
So far, many tracking approaches have been developed, such as Kalman filter [1], mean shift [2], and Particle filter (PF) [3, 4]. Among these methods, Particle filter can solve the problem of moving object tracking in nonlinear, non-Gaussian conditions. It has a wider range of applications than Kalman filter method. Since Isard and Blake [5] introduce it into the field of video based tracking, it has become an important research direction gradually. However, in the particle filter framework, the tracking performance depends greatly on the feature selection and expression. The color feature is stable and insensitive to object deformation, rotation, and partial occlusion, and it has advantages of simple calculation and fast processing, so it has been widely used in object tracking. In [6], Perez et al. proposed the color-based particle filter algorithm for object tracking and extension to multiple objects. Nummiaro et al. [7] proposed weighted color histogram observation model in RGB space and the object model is updated adaptively. But, color histogram is sensitive to illumination and environmental interference. Therefore scholars put forward many improved algorithm. Reference [8–10] constructed the object observation model by using spatiogram, which is histogram augmented with spatial means and covariance to capture a richer description of the object. Spatiogram contains color information and spatial layout of these colors and has better robustness than color histogram. Reference [11] constructed the multiple color distributions model according to prior knowledge of the object; it has a good performance especially when the object colors changed drastically. These algorithms above improved the color feature performance in some degree but did not consider the influence of the illumination. For this reason, many scholars have proposed multiple cues tracking method, such as two features fusion: color fused with texture [12], or edge [13, 14], sift [15], and motion [16], and more than two features fusion: color fused with edge, texture, regional gray feature, and so on [17, 18]. Multiple cues fusion method improves the robustness of object model and overcomes the instability brought by using a single measurement source, but the extraction of a variety of feature and the calculation of the fusion strategy can lead to large sample space, heavy calculation burden. So the multiple cues fusion method cannot meet the real-time requirement in the actual project.
On account of the problems above, this paper starts from improving the descriptive ability of traditional color feature model, introduces the local entropy into the object feature model, proposes an object tracking algorithm based on color local entropy particle filter (CLEPF). First, the weight function of the histogram is improved by using a scale factor. Second, aiming at the disadvantages of the color feature which is sensitive to illumination and susceptible to environmental interference, we map the object from color feature space to local entropy feature space, a color local entropy object observation model is constructed in Hue-Saturation-Value (HSV) color space. Additionally, an adaptive update strategy of the object template is designed, and the number of particle is adjusted dynamically according to the tracking performance. The experimental results show that the proposed algorithm is more effective and robust by comparison with several existing algorithms.
The organization of this paper is shown as follows. Section 2 explains the theory of particle filter algorithm briefly. Section 3 gives the definition of color local entropy (CLE) and constructs the color local entropy object observation model. Section 4 designs a particle filter algorithm based on color local entropy, including the selection of the motion model, the adaptive updating strategy of object template, the dynamical adjustment of particle number, and the detailed realization of the algorithm. The experimental data and results are analyzed and discussed in Section 5. Section 6 is the conclusion.
2. Particle Filter Algorithm
In the particle filter algorithm, the posterior density of the system state is described by a group of weighted particles
The standard particle filter algorithm procedure is as follows.
Particle set initialization:
Importance sampling: take new particle samples from importance probability density function
Calculate system posterior probability density:
Resample: count effective particle number
Output: system state estimation
3. Color Local Entropy Object Model
Feature selection and expression is the key step of object tracking. Good feature model should have good stability, intuition and can distinguish the object from its background. The color histogram is insensitive to object deformation, rotation, scaling, and partial occlusion and has the advantages of easy calculating and fast processing. But it is sensitive to illumination variation. For this question, the local entropy of the image reflects the amount of the image local information, which has advantages of well resisting radiation distortion, deformation, and illumination change; it has been successfully applied in the image matching and retrieval [21–23]. Therefore, this paper combines color weight histogram feature with local entropy and builds a new color local entropy object observation model.
3.1. Weight Color Histogram Feature
Because the peripheral pixels may belong to the background or be affected by partial occlusion, the pixels which near center of region have more identification value than peripheral pixels. Therefore, to increase the reliability of the color distribution, pixels farther from the center are given less weight than pixels near the center by employing a weighting function. In traditional method, weight is Gaussian function [7], which only gives one pixel that in the center of region maximum weight. In this paper we improve the traditional weighting function by defining a scale factor ∂. The proposed method can give pixels maximum weight within a certain range of the object center by adjusting the scale factor ∂; it can reflect the difference of the pixels in different position and describe the object model accurately.
where

The object area schematic diagram.
3.2. Image Local Entropy
Image local entropy is widely used in image matching and retrieval for its advantages of resisting geometric distortion, easy calculation, and stability to illumination. Let an image which size is
Local entropy is based on statistical methods, which reflects the amount of the local image information. Although in image matching and retrieval field local entropy has unique advantages, it can be seen from formula (6) that the definition of probability
3.3. Color Local Entropy Object Model
This paper combines weight color histogram and the local entropy, complementary advantage proposes color local entropy (Color Local Entropy, CLE) model.
where
The CLE can describe and express the object better than traditional color feature model. On one hand, CLE not only retains the advantages of the color model which is stable to object deformation and partial occlusion, but also has good robustness to illumination variations; meanwhile the introduction of the weight function which represents space position information reduces the influence of the symmetry of entropy. On the other hand, analyzing from the view of human vision, the more concentrated the object color distributes in the space, the more the stimulation that human eyes get, and vice versa [24]. For CLE proposed, the value of the CLE indicates the decentralization or centralization of color distribution; thus compared with the traditional color model, CLE is more in accord with the characteristics of the human vision.
Taking the logarithm operation in (7) into account, discarding the higher order term of the Taylor series expansion, the following approximate formula is obtained:
In the tracking process, let
4. Object Tracking Algorithm Based on CLEPF
4.1. Object Motion Model
This paper uses external rectangle to describe the state of the moving object. Let
Since the time of the frame interval is very short, it is assumed that the moving object is in uniform velocity within a frame interval. Therefore, this paper chooses the first-order autoregressive model. The state motion model is defined as
4.2. Object Template Updating and the Determination of Particle Number
During the tracking process, because of the object deformation and object occlusion, the fixed object reference template cannot adapt to the change of object and environment. With error continuous accumulating, the tracking accuracy will decline, so it is necessary to update the object observation model to ensure the tracking accuracy and stability.
Let
If
If
4.3. CLEPF Tracking Algorithm Steps
The complete CLEPF algorithm for object tracking is summarized as follows.
Initialize particle: set = 0;
Object state prediction: predict the state of
Calculate particles weight: weights of the particles are calculated and normalized by formula (2).
Update object model and set the particle number: update object reference model and adjust particle number using the updating strategy in Section 4.2.
Output the tracking result: let the particle with the highest weight denote the tracking result and then acquire the tracking area.
Particle resample: resample the particle according to importance principle [19]. Set
5. Experiment Results and Analysis
To verify the accuracy and robustness of the proposed algorithm (CLEPF), considering object occlusion, illumination variation, small camera vibration, and some other factors, three typical comparison experiments have been designed to test the proposed algorithm and compared with three representative tracking method: color histogram [7], spatiogram [8], color and edge histogram [14]. In the end, we evaluate and compare the algorithms' quantitative performance and empirically demonstrate the accuracy of the proposed method.
The experimental environment is based on the PC with Intel Core 2 processor, XP operation system, and 1 G RAM. Software development environment is VC++2010. The initial object model was selected manually. HSV color space, which is simplified form of Munsell color space, is in accordance with human's perception, and its three components are relative independent. Therefore H-S histogram (16 × 4) is constructed in the HSV color space. All the resolution of experimental videos is 320 × 240. The parameters are set as follows:
5.1. Vehicle Tracking
The first experimental video is taken from video surveillance system in Beijing Institute of Technology. The environmental background contains building, pedestrians, and other moving vehicles. The vehicle tracking results are shown in Figure 2. From Figure 2(a) we can see that the color-based particle filter can keep tracking the white car, but the estimated object area deflect from the real object center. The tracking results of CLEPF are shown in Figure 2(b), it is clear that CLEPF has a much better performance in the tracking accuracy as well as adaptability of the object deformation. Overall, both algorithms can accomplish the tracking task under uncomplicated environment.

Comparison of tracking results for experiment 1 (the number in the upper left corner is the frame number).
5.2. Object Tracking under Object Occlusion
The second set of experiment is used to verify the robustness of our algorithm under the condition of object occlusion. In this experiment, the object is a pedestrian, which had elastic motion and morphological change, moved from right to left. In the process of walking, the pedestrian was occluded by a quickly passed cyclist, a lamppost, and a tree. The tracking results are shown in Figure 3. From Figures 3(a) and 3(b), we can see that the color histogram method lost the object, and spatiogram method also cannot work very well; the tracking window gradually becomes larger during the tracing process. The pedestrian has been occluded three times, so color information and edge information are all disturbed in some degree. Therefore, as shown in Figure 3(c), the fusion algorithm performance is not very well. Because of using the proper model updating strategy and adaptive particle number method, from Figure 3(d), it can be seen that the proposed method could track the object correctly even if another object occludes the object partially or fully.

Comparison of tracking results for experiment 2 (the number in the upper left corner is the frame number).
5.3. Object Tracking under Illumination Variation
The third section of video sequence was captured by a hand-held camera in a typical laboratory environment. During the shoot, small camera dithering was unavoidable. This experiment is designed to verify the tracking robustness of our algorithm (CLEPF) under the condition of illumination change. The background illumination is changed by turning off and turning on the electric light repeatedly. As shown in Figure 4, the background illumination of Frame 45 is much darker than Frame 25 by turning off the light. Then, because of the camera autodimming function, the background brightness of Frame 75 is a little bit higher than Frame 45. Next, turn on the light again. According to the experimental results in Figure 4, although the background illumination changes again and again, CLEPF and the fusion algorithm work well and have a stable object tracking capability, color histogram, and spatiogram method lost the object.

Comparison of tracking results for experiment 3 (the number in the upper left corner is the frame number).
In addition, tracking errors of those methods are computed and compared in Figure 5 for experiment 2 and experiment 3. Tracking error is the distance between the real object center position (obtained manually) and the centroid of object tracking window. As shown in Figure 5 our algorithm shows the good robustness and tracking accuracy. Tables 1 and 2 show the average runtime per frame and average error of four algorithms for experiment 2 and 3. From Tables 1 and 2, it is clear that the color and edge method take the longest time. That is because the extraction of multiple features and the calculation of the fusion strategy affect the computing speed, and other three methods take approximately the same time. Although the color histogram method takes the least time compared to other methods, it has the lowest accuracy. It is worth noticing that the proposed algorithm takes more time in experiment 2 than in experiment 3. That is because in experiment 2, three times object occlusion caused the increase of particle number and then affected the speed of the algorithm. Overall, taking into consideration of the run time and accuracy factors, our algorithm has better performance than other algorithms and meets the requirement of real-time as well.
Performance comparison of four algorithms for experiment 2.
Performance comparison of four algorithms for experiment 3.

Comparison of the error curves.
6. Conclusion
This paper proposed a robust and efficient object tracking algorithm. The contributions of this paper can be summarized as follows.
We improved the histogram weighted function.
A new color local entropy object observation model is constructed in HSV color space by mapping the object from color feature space to local entropy space.
An adaptive strategy of the object template is designed to update the appearance template online, and the number of particles is adjusted dynamically according to the tracking performance.
Compared with several existing methods, it is clear that the proposed algorithm is more effective and robust under the condition of object occlusion, illumination variation, and small camera dithering. Furthermore, the tracking speed of our algorithm is fast enough to meet the real-time requirement of practical engineering application.
Our algorithm can work well and have a good performance in object tracking, but there are still problems in it. Such as, when the illumination changes dramatically, the tracking error will increase, and object observation model is determined by global feature only. On the premise of meeting the real-time quality, how to integrate the local feature with global feature, further improving the robustness of the algorithm is what we need to do in future research.
Footnotes
Acknowledgment
The authors acknowledge the support from the National Natural Science Foundation of China (Grant no. 61375100).
