Sage Journals: Discover world-class research

Abstract

Pedestrian detection and tracking is the key to autonomous vehicle navigation systems avoiding potentially dangerous situations. Firstly, the probability distribution of colour information is established after a pedestrian is located in an image. Then the detected results are utilized to initialize a Kalman filter to predict the possible position of the pedestrian centroid in the future frame. A Camshift tracking algorithm is used to track the pedestrian in the specific search window of the next frame based on the prediction results. The actual position of the pedestrian centroid is output from the Camshift tracking algorithm to update the gain and error covariance matrix of the Kalman filter. Experimental results in real traffic situations show the proposed pedestrian tracking algorithm can achieve good performance even when they are partly occluded in inconsistent illumination circumstances.

Keywords

Autonomous Vehicles Pedestrian Tracking Kalman Filter Camshift Algorithm Colour Probability Distribution

1. Introduction

In road traffic accidents, pedestrians are more likely to be injured than drivers and passengers because of poor pedestrian protection countermeasures. According to a World Health Organization (WHO) report, road traffic injuries were the ninth leading cause of death globally in 2011 and will be the fifth in 2030 unless certain actions are undertaken [1]. Fortunately, the United Nations General Assembly has adopted a resolution named the Decade of Action for Road Safety (2011–2020) to stabilize and reduce road deaths [2]. The requirement to enhance pedestrian safety is particularly urgent in developing countries, such as China, because of the high distribution of the pedestrian fatalities. Naci et al. [3] indicate that the magnitude of pedestrian fatalities varies across low-, middle- and high-income countries. Due to the lack of automotive safety interventions and strict traffic law enforcement in developing countries, pedestrians tend to be at a higher risk than drivers and passengers in a traffic accident [4]. According to a report from the Governors Highway Safety Association (GHSA), the number of pedestrians killed on United States in 2014 is 4,884, which is 19 percent higher than that in 2009. As in China, pedestrians are most likely to be killed followed by vehicle passengers, motorcyclists, bicyclists and vehicle drivers [5].

To reduce the pedestrian-vehicle accidents and protect the safety of pedestrians effectively, strict pedestrian protection regulations are put forward by governments and certain countermeasures are conducted by automotive companies and research departments. Most of all, the characteristics of pedestrian-vehicle crashes and pedestrian behaviour are analysed to develop suitable systems for pedestrian protection. For example, Koh et al. [6] examined the crossing behaviour of pedestrians at signalled pedestrian crossings to further enhance their safety. Li et al. [7] conducted an investigation of minibus-pedestrian collisions to better understand the injury patterns and injury risk of pedestrians in China. With automotive safety technology, the active pedestrian protection system can help the driver detect pedestrians in front of the host vehicle and sense a crash before it happens. If the situation is dangerous, the pedestrian protection system can stop or decelerate the vehicle. It can also trigger the passive pedestrian protection system to reduce the severity of or even avoid a crash [8]. For example, Broggi at al. [9] presented an automatic braking application for a pedestrian detection system aimed at localizing potentially dangerous situations in critical urban scenarios. Dollar et al. [10] declared that automatically detecting pedestrians from moving vehicles could have a considerable economic impact and the potential to substantially reduce pedestrian injuries and fatalities. Fredriksson et al. [11] confirmed that an integrated pedestrian protection system combining active and passive countermeasures has a potential to better enhance pedestrian safety.

The paper is organized as follows. Section 2 reviews the literature concentrating on pedestrian tracking and presents the main purposes of the paper. Section 3 describes the pedestrian tracking methodology. Case study results of pedestrian tracking experiments are presented in Section 4. Section 5 concludes the paper.

2. Related Work

Pedestrian detection and tracking is key to pedestrian protection systems and has attracted lots of attention in recent years using different kinds of passive and active sensors, such as video cameras, infrared cameras, radar and lidar scanners [12 –14]. Compared with other kinds of sensors, the video sensor is the main solution for the pedestrian protection application because of its lower price, better signal-to-noise ratio and more amounts of information. Pedestrian tracking aims to locate instances of each detected pedestrian in the frames of the analysed film [15]. After being detected, the target pedestrian should be tracked to narrow the detection area in the next frame. Moreover, the tracking results can provide the pedestrian's movement trajectory, which is essential to predict the motion of the pedestrian and estimating the probability of collision [16].

Current pedestrian tracking methods can be categorized into four classes: model, region, active contour and feature based. Model-based tracking algorithms track pedestrians by matching projected linear, 2D or 3D pedestrian models, produced with prior knowledge, to image data. For example, Munder et al. [17] combined a generative shape model represented by a set of linear subspace models and a discriminative texture classifier to achieve integrated pedestrian detection and tracking. Generally, a more complex human body model means a more accurate tracking result. Unfortunately, the computation is also more expensive. Region-based pedestrian tracking methods fulfil the tracking tasks by matching and searching for the variations of the image regions corresponding to the moving targets. Region-based tracking algorithms require small deformations and cannot reliably handle occlusion between objects [18]. Increasing the tracking performance, colour or motion information gives more useful cues to overcome the deformations. Active contour-based tracking methods differ from region based by directly extracting outlines of targets as bounding contours and updating them dynamically in successive frames. Rathi et al. [19] provided particle filter-based movement and deformation object tracking in a geometric active contour framework, which can deal with partial occlusions and can track robustly even in the absence of a learnt model. Feature-based tracking methods need to extract global or local features of the tracked targets and categorize them into higher level features first, then match the features between adjacent image sequences [20]. The method can handle partial occlusion as long as the target feature portion is visible.

Current pedestrian-tracking methods are mainly oriented to surveillance systems, where they generally track moving objects in an image sequence captured by a static camera [21]. In pedestrian protection systems the camera is mounted on the vehicle and moves with it, which may lead to difficulties in handling complex pedestrian appearance changes caused by factors such as illumination variation, partial occlusion and camera motion. This paper proposes a technique aiming to improve real-time pedestrian tracking performance in an urban traffic environment even with partial occlusions. Based on the concept of a region-based object tracking theory, a robust pedestrian-tracking method integration of Camshift and Kalman filter algorithms is presented. The traditional Camshift has the ability to detect and track the target robustly because the colour distribution map of the target shows slow variation. However, it easily fails to track the target when occlusion or similar colour interference appears because of the change of the colour probability distribution [22]. Therefore, the Kalman filter has been used to improve these inaccurate tracking results by predicting the position of the target in the next frame using its motion information.

3. Methodology

3.1. Overview of the pedestrian protection system

Pedestrian detection and tracking are the two main modules of our pedestrian protection system based on monocular machine vision, as shown in Fig. 1.

Figure 1.

Framework of the pedestrian protection system

Once the system is initialized, the processing unit captures the images in front of the host vehicle and starts the pedestrian detection module utilizing the pedestrian classifier, trained offline. The pedestrian detection module is fulfilled by combining the AdaBoost and support vector machine (SVM) to train the pedestrian classifier, which is not the main purpose of this paper and has been introduced in previous work [23]. If a pedestrian is detected successfully in three consecutive frames, the tracking state is assigned to be true and the pedestrian tracking module is triggered. In the pedestrian tracking module, which is the main work reported in this paper, the detected pedestrian is tracked and their motion and colour information is recorded to analyse their trajectory. Fig. 2 displays the flowchart of the pedestrian-tracking module.

Figure 2.

Flowchart of the pedestrian tracking module

The Kalman filter is adopted to predict the position of the detected pedestrian in the next frame. Then, combined with the colour probability distribution map of the detected pedestrian, the Camshift algorithm is utilized to track the pedestrian in the search window according to the position-prediction results. If the pedestrian is tracked successfully their current position is updated, as well as the state and the error covariance matrices of the Kalman filter. If the target pedestrian is not tracked successfully in three consecutive frames, it is believed that they have moved out of sight or the tracking is invalid. Then the pedestrian detection module starts again, to find another pedestrian.

3.2. Constructing colour probability distribution of pedestrians

Commonly, once the pedestrian is located their colour characteristics are less affected by the variation of their motion. Therefore, the tracking of the detected pedestrian and adjusting of their size can be fulfilled based on their colour probability distribution. RGB (red-green-blue) and HSV (hue-saturation-value) colour are the two classic image colour spaces in machine vision applications. The components of the RGB colour space are sensitive to changes in brightness. HSV colour space uses hue, saturation and brightness to describe the colour quantitatively, reflecting the human visual observation of colour intuitively. Considering the fact that hue, saturation and brightness are separate in HSV colour space, the stability of the algorithm will increase when using the HSV space [24]. Therefore, the hue value of the pedestrian colour image, which is symbolized by H∈[0,360), is utilized to build the probability distribution and to enhance pedestrian tracking stability. The transformation equations from the RGB space to HSV space of the image are:

\begin{array}{l} H = {\begin{matrix} 60 \times [\frac{g - b}{m a x (r, g, b) - m i n (r, g, b)}], \begin{matrix} i f (r = m a x (r, g, b)) \end{matrix} \\ 60 \times [\frac{b - r}{m a x (r, g, b) - m i n (r, g, b)} + 2], \begin{matrix} i f (g = m a x (r, g, b)) \end{matrix} \\ 60 \times [\frac{r - g}{m a x (r, g, b) - m i n (r, g, b)} + 4], \begin{matrix} i f (b = m a x (r, g, b)) \end{matrix} \end{matrix} \\ S = \frac{m a x (r, g, b) - m i n (r, g, b)}{m a x (r, g, b)} \\ V = m a x (r, g, b) \end{array}

(1)

where H, S and V represent the hue, saturation and brightness value of each pixel respectively and r, g and b represent the red, green and blue value of each pixel respectively.

To find the colour histogram of the detected pedestrian, the value of the image hue is divided into several small bins then the total number of hues that fall within the range of each bin is computed. Let the total number of image pixels be n₁, the number of bins of hue be s, with x_i the position each image pixel and the histogram index function of x_i is c(x_i), then the object histogram of the hue can be described as:

q_{u} = \sum_{i = 1}^{n_{1}} δ [c (x_{i}) - u], \begin{matrix} u = 1, 2, …, s \end{matrix}, \begin{matrix} i = 1, 2, \dots, n_{1} \end{matrix}

(2)

where δ is a Kronecker Delta function and q_u is the hue histogram corresponding to level u. Considering the real-time requirements and pedestrian tracking performance, the quantitative level is divided into 48 bins, i.e., s=48.

To emphasize the main colour information of the target pedestrian, values of hue histogram lower than a certain threshold are eliminated to keep only those with high values. Set the maximum value of the hue histogram as max(q_u) and the threshold value as T=max(q_u)/ $\sqrt{2}$ , and make the hue histogram which has a value lower than the threshold T zero. Then the colour probability distribution of the target pedestrian can be achieved by substituting each pixel of the original image with its hue histogram, which can be described as:

p_{u} = \min (\frac{255}{\max (\hat{q})} {\hat{q}}_{u}, 255), \begin{matrix} u = 1, \dots, s \end{matrix}

(3)

where p_u is the value of the colour probability distribution map and is resized to make them fall in the range of [0, 255]. After the above transformation, each pixel of the pedestrian image is represented by its hue histogram value. A higher value of the pixel means it is more possible to see the distribution of this bin of hue value in the target image. Fig. 3 shows the hue histogram and colour probability distribution map with a pedestrian in the image.

Figure 3.

Hue histogram and probability distribution map of a pedestrian in the image

3.3. Position prediction based on the Kalman filter

The Kalman filter was first put forward by R.E. Kalman in 1960 when attempting to describe a recursive solution to the discrete-data linear filtering problem [25]. The time update and measurement-update are the main equations of the Kalman filter. The time update equation aims to find a priori estimates for the next time step by projecting forward the current state and error covariance estimates. The measurement update equations can obtain an improved a posteriori estimate by incorporating a new measurement into the a priori estimate.

If a system is linear and its process noise and measurement noise are Gaussian white noise, it can be modelled as a pair of linear stochastic process and measurement equations [26]. The state equation of the system at time step k can be defined as:

X_{k} = Φ X_{k - 1} + w_{k - 1}

(4)

where X∈Rⁿ is the system state vector, X_k is the system state at time step k, Φ is an n×n state transition matrix that transfers the system state from the previous time step to the current step k in the absence of either a driving function or process noise and w_k is the process noise with a zero mean Gaussian white noise random sequence, whose covariance matrix is Q.

The system state can be achieved indirectly from the measurement value. Take Z_k∈R^m as the system measurement vector. Then the relationship of the state and measurement vector can be described as:

Z_{k} = Ψ X_{k} + v_{k}

(5)

where Z_k is the measurement vector at step k, Ψ is an m×n measurement matrix that relates the current state X_k to the measurement Z_k and v_k is the measurement noise with a zero mean Gaussian white noise random sequence, whose covariance matrix is R.

The a priori estimates for the next time step can be achieved by the prediction step of the Kalman filter, which is realized by a set of time update equations as:

{\begin{cases} {\hat{X}}_{k} (-) = Φ {\hat{X}}_{k - 1} (+) \\ P_{k} (-) = Φ P_{k - 1} (+) Φ^{T} + Q \end{cases}

(6)

where ${\hat{X}}_{k} (-)$ is a priori state estimate and $P_{k} (-)$ is the a priori estimate error covariance at time step k.

The correction step is carried out by a set of error covariance prediction and correction equations as:

{\begin{cases} K_{k} = P_{k} (-) Ψ_{}^{T} / [Ψ P_{k} (-) Ψ_{}^{T} + R] \\ {\hat{X}}_{k} (+) = {\hat{X}}_{k} (-) + K_{k} [Z_{k} - Ψ {\hat{X}}_{k} (-)] \\ P_{k} (+) = (I - K_{k} Ψ) P_{k} (-) \end{cases}

(7)

where K_k is the n×m Kalman gain matrix at time step k, ${\hat{X}}_{k} (+)$ is a posteriori state estimate according to the actual measurement Z_k and the predicted measurement $Ψ {\hat{X}}_{k} (-)$ at time step k and $P_{k} (+)$ is a posteriori estimate error covariance at time step k.

3.4. Pedestrian tracking based on Camshift

The Camshift is a moving object tracking algorithm based on the change of colour probability distribution in an image sequence. Its core algorithm is the Meanshift, which is a non-parametric probability density estimation method [27]. Unlike the Meanshift, the Camshift is designed for dynamically changing distributions. It can implement tracking moving objects in continuous video frames through adaptively adjusting the size and position of the search window, which helps save search time and improve efficiency. The detailed steps for pedestrian tracking based on the Camshift are as follows:

Calculate the colour probability distribution map of the predicted search window and obtain the first order and zero order moments by:

{\begin{cases} M_{00} = \sum_{x} \sum_{y} I (x, y) \\ M_{10} = \sum_{x} \sum_{y} x I (x, y) \\ M_{01} = \sum_{x} \sum_{y} y I (x, y) \end{cases}

(8)

where (x, y) are the coordinates of the pixel and I(x, y) is the colour probability distribution map of the search window.

Calculate the mass centre (x_c, y_c) of the search window:

{\begin{cases} x_{c} = \frac{M_{10}}{M_{00}} \\ y_{c} = \frac{M_{01}}{M_{00}} \end{cases}

(9)

Set the centre of the window as the initial Meanshift algorithm and repeat steps 1 and 2 until the mass centre position converges to a point. Take the results as the pedestrian position at current time k and update the Kalman filter parameter, as well as the pedestrian colour probability distribution map.

Unlike the traditional Camshift that needs to determine the initial search window manually, the pedestrian detection module can locate the pedestrian accurately and the tracking module can predict the possible position of the detected pedestrian in future frames. It can automatically initialize the search window and track multiple objects at the same time. With the prediction of the Kalman filter and narrowed search window, the Camshift will find the optimal position in this window. At the same time, the output of the Camshift can provide the observation value to modify the prediction value in the next frame, which can speed up the convergence rate of the tracking process and overcome the occlusion problem effectively.

4. Case Study

To verify the performance of the proposed pedestrian-tracking algorithm, an intelligent vehicle prototype named DLUTIV-I is taken as the test platform. The system is run on a Core2 2.66GHz PC that exhaustively searches a 320×240 image captured by an AVT Stingray camera [28]. The pedestrian classifiers are trained offline using the samples manually selected from different kinds of circumstances and loaded by the system to realize the real-time pedestrian detection. The pedestrian detection results are taken to initialize the parameters of the Kalman filter.

The state vector of the Kalman filter includes the centroid pixel position of the detected pedestrian and their position velocity. Let (x_k, y_k) represent the centroid pixel position of the pedestrian and (△x_k, △y_k) be the position velocity at time step k in the horizontal and vertical directions, respectively. The measurement vector includes the centroid real pixel position of the pedestrian at the current step. Then the pedestrian state and measurement vector at time step k can be represented as:

\begin{array}{l} X_{k} = {(x_{k}, y_{k}, Δ x_{k}, Δ y_{k})}^{T} \\ Z_{k} = {(x_{k}, y_{k})}^{T} \end{array}

(10)

Taking into account the fact that the speed and direction of the pedestrian motion does not change significantly, it is assumed that the motion is uniform in a straight line during two consecutive frames, that is:

{\begin{cases} x_{k} = x_{k - 1} + Δ t * Δ x_{k - 1} \\ y_{k} = y_{k - 1} + Δ t * Δ y_{k - 1} \\ Δ x_{k} = Δ x_{k - 1} \\ Δ y_{k} = Δ y_{k - 1} \end{cases}

(11)

Then the system-state transition matrix and the measurement matrix can be written as:

Φ = [\begin{array}{l} 1 0 Δ t 0 \\ 0 1 0 Δ t \\ 0 0 1 0 \\ 0 0 0 1 \end{array}] \begin{matrix} , \end{matrix} Ψ = [\begin{array}{l} 1 0 0 0 \\ 0 1 0 0 \end{array}]

(12)

where △t is the time interval of the pedestrian protection system. When the system works at the speed of 25 frames per second △t=40ms.

To realize pedestrian tracking, the Kalman filter should be initialized first. The tracking procedure is activated after the pedestrian is detected successfully in three consecutive frames. Suppose the consecutive two frames to be at time step k and k+1, the initial state vector can be described as:

X_{0} = {[x_{t + 1}, y_{t + 1}, (x_{t + 1} - x_{t}), (y_{t + 1} - y_{t})]}^{T}

(13)

We also need to specify the initial covariance matrix P₀ for initial state X₀. Since P_k is updated iteratively as more images are processed, it can be initialized to a large value. Suppose the predicted centroid position has a ±5 pixel deviation from the real position in both horizontal and vertical directions and the position variation velocity has an ±8 pixel deviation. Then the initial error covariance matrix P₀ can be written as:

P_{0} = [\begin{matrix} 5^{2} & 0 & 0 & 0 \\ 0 & 5^{2} & 0 & 0 \\ 0 & 0 & 8^{2} & 0 \\ 0 & 0 & 0 & 8^{2} \end{matrix}]

(14)

Based on the observation of the pedestrian motion, we set the standard deviation for the centroid position as 10 pixels and 15 pixels its position variation velocity. So the state and measurement noise covariance matrices can be described as:

\begin{array}{l} Q = [\begin{matrix} 5 & 0 & 0 & 0 \\ 0 & 5 & 0 & 0 \\ 0 & 0 & 6 & 0 \\ 0 & 0 & 0 & 6 \end{matrix}] \\ R = [\begin{matrix} 2 & 0 \\ 0 & 2 \end{matrix}] \end{array}

(15)

Using the state prediction and correction equations in conjunction with the initial conditions, the state vector ${\hat{X}}_{k} (-)$ and its covariance matrix $P_{k} (+)$ are estimated at each frame. The covariance matrix $P_{k} (+)$ indicates the uncertain range of pedestrian location.

During the Camshift searching procedure, the Bhattacharrya coefficient is used to define whether the centroid position converges at a certain point [29]. When searching the centroid, the regular movement of legs may accumulate the error of the centroid position after several frames. Commonly, the pedestrian is composed of a head, torso and legs. Once a pedestrian is located, the colour probability distribution of their torso changes less than that of their legs. Compared with the colour probability distribution based on the whole pedestrian, that based on the torso is more distinct, as shown in Fig. 4. Therefore, the colour probability distribution based on the torso is taken as the input for the Camshift.

Figure 4.

Colour probability distribution based on different human parts

Fig. 5 shows the tracking image sequences with pedestrians moving in front of a static experimental vehicle in an inconsistent illumination situation. The pedestrians are located using the pedestrian classifier trained offline and their locations are transferred to the pedestrian module. The pedestrian centroid tracking result indicates the moving trajectory of the pedestrian in the image, as shown in Fig. 6.

Figure 5.

Tracking image sequences of pedestrians in front of a static vehicle

Fig.7 shows the tracking image sequences with pedestrians moving in front of a vehicle that moves at an average speed of 18km/h. The pedestrians are moving in front of the vehicle at normal speed and maintaining direction. After detecting a pedestrian in the image, they are tracked successfully until they are out of sight after 210 consecutive frames. After that, the system judges that the tracking fails and the pedestrian detection module is restarted. At the 318th frame, a new pedestrian is detected and then is tracked steadily until the 432nd frame, when a new pedestrian shows up. After the two pedestrians are verified by the pedestrian detection module, they are tracked until one moves out of sight at the left of the image and the other one is tracked to the end of the experiment. The tracking results are shown in Fig. 8. The experiment results show that the proposed pedestrian detection and tracking method is effective.

Figure 6.

Tracking results with pedestrians moving in front of a static vehicle

Figure 7.

Tracking image sequences with pedestrians moving in front of a running vehicle

Figure 8.

Tracking results with pedestrians moving in front of a running vehicle

5. Conclusions

Pedestrians are the weakest and most at-risk group when road crashes happen. The purpose of this paper is to present a technique to make autonomous vehicles safer by presenting a novel pedestrian tracking method combining the Camshift algorithm and the Kalman filter. Hue histogram and probability distribution help enhance the stability of the pedestrian-tracking module. The tracking method based on colour probability distribution can speed up the convergence rate of the tracking process and overcomes the occlusion problem. Moreover, when conducting the Camshift tracking algorithm, the colour probability distribution of the torso is more distinct than that of the pedestrian as a whole. Therefore, the colour probability distribution of the torso is chosen as the input to the Camshift algorithm, which can reduce the accumulated error after several frames.

Considering that the speed and direction of pedestrian motion do not change significantly during two consecutive frames, the Kalman filter is initialized after the pedestrian is located and the future position of the pedestrian can be predicted. The Camshift is utilized to track the pedestrian in a specific search window in the next frame based on the prediction results, avoiding tracking throughout the whole image; the tracking robustness is improved as well. Several experimental scenarios were implemented in actual traffic situations. Results show that the proposed method can track multiple pedestrians in a complex scenario. It can also overcome problems such as sudden appearance, disappearance and inconsistent illumination of pedestrian scenes. The algorithm provided basic research for collision-avoidance technology for autonomous vehicles.

Footnotes

6. Acknowledgements

This work was mainly supported by the National Natural Science Foundation of China under Granted No. 51575079 and 51305065, as well as the Fundamental Research Funds for the Central Universities under Granted No. DUT16QY10. Finally, the authors are grateful for the reviewers and the editor for their insightful comments, which helped improve the quality of the paper.

References

World Health Organization. Violence and Injury Prevention & World Health Organization. Global status report on road safety 2013: Supporting a decade of action, 2013.

Sleet

Baldwin

Dellinger

Dinh-Zarr

The decade of action for global road safety. Journal of Safety Research 2011; 42: 147–148.

Naci

Chisholm

Baker

. Distribution of road traffic deaths by road user group: A global comparison. Injury Prevention 2009; 15: 55–59.

Nordfjaern

Rundmo

Perceptions of traffic risk in an industrialised and a developing country. Transportation Research Part F-Traffic Psychology and Behaviour 2009; 12: 91–98.

Zhang

Yao

Cui

Xiang

. Basic characteristics of road traffic deaths in china. Iranian Journal of Public Health 2013; 42: 7–15.

Koh

Wong

Chandrasekar

. Safety evaluation of pedestrian behaviour and violations at signalised pedestrian crossings. Safety Science 2014; 70: 143–152.

Fan

Yin

. Pedestrian injury patterns and risk in minibus collisions in china. Medical Science Monitor 2015; 21: 727–734.

Froming

Schindler

Kuhn

Requirement engineering for active safety pedestrian protection systems based on accident research In: 10th International Forum on Advanced Microsystems for Automotive Applications, 2006, pp.79–106.

Broggi

Cerri

Ghidoni

Grisleri

Jung

. A new approach to urban pedestrian detection for automatic braking. IEEE Transactions on Intelligent Transportation Systems 2009; 10: 594–605.

10.

Dollar

Wojek

Schiele

Perona

Pedestrian detection: An evaluation of the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence 2012; 34: 743–761.

11.

Fredriksson

Rosen

Integrated pedestrian countermeasures - Potential of head injury reduction combining passive and active countermeasures. Safety Science 2012; 50: 400–407.

12.

Gandhi

Trivedi

. Pedestrian protection systems: Issues, survey, and challenges. IEEE Transactions on Intelligent Transportation Systems 2007; 8: 413–430.

13.

Krotosky

Trivedi

. On color-, infrared-, and multimodal-stereo approaches to pedestrian detection. IEEE Transactions on Intelligent Transportation Systems 2007; 8: 619–629.

14.

Navarro-Serment

Mertz

Hebert

Pedestrian detection and tracking using three-dimensional LADAR data. International Journal of Robotics Research 2010; 29: 1516–1528.

15.

Barbu

Pedestrian detection and tracking using temporal differencing and HOG features. Computers & Electrical Engineering 2014; 40: 1072–1079.

16.

Philomin

Duraiswami

Davis

Pedestrian tracking from a moving vehicle In: Proceedings of the IEEE Intelligent Vehicles Symposium 2000, pp. 350–355.

17.

Munder

Schnorr

Gavrila

. Pedestrian detection and tracking using a mixture of view-based shape-texture models. IEEE Transactions on Intelligent Transportation Systems 2008; 9: 333–343.

18.

Tan

Wang

Maybank

A survey on visual surveillance of object motion and behaviors. IEEE Transactions on Systems Man and Cybernetics Part C-Applications and Reviews 2004; 34: 334–352.

19.

Rathi

Vaswani

Tannenbaum

Yezzi

Tracking deforming objects using particle filtering for geometric active contours. IEEE Transactions on Pattern Analysis and Machine Intelligence 2007; 29: 1470–1475.

20.

Prioletti

Mogelmose

Grisleri

Trivedi

Broggi

Moeslund

. Part-based pedestrian detection and feature-based tracking for driver assistance: Real-Time, Robust Algorithms, and Evaluation. IEEE Transactions on Intelligent Transportation Systems 2013; 14: 1346–1359.

21.

Smeulders

Chu

Cucchiara

Calderara

Dehghan

Shah

Visual tracking: An experimental survey. IEEE Transactions on Pattern Analysis and Machine Intelligence 2014; 36: 1442–1468.

22.

Chen

Luo

Qiu

. Real-time visual object tracking via Camshift-based robust framework. International Journal of Fuzzy Systems 2012; 14: 264–271.

23.

Guo

Zhang

Zhao

. Pedestrian detection for intelligent transportation systems combining AdaBoost algorithm and support vector machine. Expert Systems with Applications 2012; 39: 4274–4286.

24.

Guanglong

Zhang

Human-manipulator interface using hybrid sensors with Kalman filters and adaptive multi-space transformation. Measurement 2014; 55: 413–422.

25.

Kalman

. A new approach to linear filtering and prediction problems. Journal of Fluids Engineering 1960; 82: 35–45.

26.

Zhang

Welch

Bishop

Huang

. A two-stage Kalman filter approach for robust and realtime power system state estimation. IEEE Transactions on Sustainable Energy 2014; 5: 629–636.

27.

Liang

Hou

Xiang

Chen

. An effective automatic tracking algorithm based on Camshift and Kalman filter. Proceedings of the SPIE, 2011; 8003: Id. 800316.

28.

http://1stvision.com/cameras/AVT-cameras.html#tab-8, Accessed on 15 Aug 2013.

29.

Nielsen

Generalized Bhattacharyya and Chernoff upper bounds on Bayes error using quasi-arithmetic means. Pattern Recognition Letters 2014; 42: 25–34.

Pedestrian Tracking Based on Camshift with Kalman Prediction for Autonomous Vehicles

Abstract

Keywords

1. Introduction

2. Related Work

3. Methodology

3.1. Overview of the pedestrian protection system

3.2. Constructing colour probability distribution of pedestrians

3.3. Position prediction based on the Kalman filter

3.4. Pedestrian tracking based on Camshift

4. Case Study

5. Conclusions

Footnotes

6. Acknowledgements

References