Sage Journals: Discover world-class research

Abstract

This paper proposes a human motion tracking approach for a daily life surveillance in a distributed wireless sensor network using ultrasonic range sensors. Because the human target often moves with high nonlinearity, the proposed approach applies the unscented Kalman filter (UKF) technique. A novel sensor node selection scheme at each time step considering both the tracking accuracy and the energy cost is presented. Experimental results in a real human motion tracking system show that the proposed approach can perform better tracking accuracy compared to the most recent human motion tracking scheme in the real testbed implementation.

1. Introduction

Human motion tracking is receiving increasing attention from researchers of different fields of study nowadays. The interest is motivated by a wide range of applications, such as wireless healthcare, surveillance, and human-computer interaction. A complete model of human consists of both the movements and the shape of the body. Many of the available systems consider the two modeling processes as separate even if they are very close. In our study, the movement of the body is the target.

Most of the human motion tracking systems are based on vision sensors. Recently, there has been a significant amount of work in tracking people trajectory across multiple image views. Some of the proposed approaches present systems that are capable of segmenting, detecting, and tracking people using multiple synchronized surveillance cameras located far from each other. But they try to hand off image-based tracking from camera to camera without recovering real-world coordinates [1]. Some other work has to deal with large video sequences involved when the image capture time interval is short [2]. The most recent work on vision-based people tracking systems develop wireless sensor networks with low-resolution camera to predict the trajectory of human movement [3]. However, most vision-based approaches to moving object detection are computationally intensive and costly expensive [4]. They often involve intensive real-time computations, such as image matching, background subtraction, and overlapping identification [4]. In fact, in many cases, due to the availability of prior knowledge on target motion kinematics, the intensive and expensive imaging detector array appears inefficient and unnecessary. For example, a video image consisting of 100 × 100 pixels with 8-bit gray level contains 80 kbits of data, while the position and velocity can be represented by only a few bits [5].

Instead of the centralized processing tracking system based on vision, a promising alternative system named distributed wireless sensor network (WSN) has been quickly developed recently. It consists of many low-cost, spatially dispersed position sensor nodes. Each node can compute and process information that it received and transfer the information among the sensor nodes that are placed within its communication range or to its leader node. Although there are many applications on WSN on target tracking problems [5–10], few papers can be found on human motion tracking in real-time systems [11]. We will develop such a system by WSN in this paper.

From our point of view, human tracking with multiple sensors is an intrinsic multisensor data fusion problem. Multisensor data fusion is such a process through which we combine readings from different sensor nodes, remove inconsistencies, and pull all the information together into one coherent structure. Although some work of multisensor data fusion in WSN has been proposed [6], the tracking accuracy is still limited because of the high nonlinearity property of the human target. In this paper, a UKF filter is employed to estimate the velocity and position of the human trajectory in WSN. UKF filter has the ability to switch between a high process noise (or alternatively, higher order or turn) model in the presence of maneuvers and a low process noise model in the absence of maneuvers. This point gives the UKF filter its advantage over simpler estimators like the Kalman filter and EKF. Compared to the existing work based on EKF [6], the proposed algorithm can give more accurate estimation by using multiple models for human motion in a realtime tracking system developed in this paper.

The layout of the paper is as follows. Section 2 presents the multiple models for human motion tracking. Section 3 presents the UKF estimator for our application. Section 4 proposes the sensor node selection method for our framework. Section 5 presents the simulation results and experimental results. Conclusions and future work are drawn in Section 6.

2. Problem Formulation

In this section, we formulate the human motion tracking as a distributed multisensor data fusion problem. We consider the human moving in a 2D Cartesian coordinate system. The target state includes the human velocity, the human position in the coordinate, and the turn rate when the trajectory is along a curve. We can build up the system models as follows.

2.1. Coordinated Turn Model

In order to describe the human's more complex trajectory, such as turn left or turn right, here we apply the coordinated turn model similar to that in [5]:

\begin{matrix} x (k + 1) = F (x (k)) + G v (k), \\ where x (k + 1)  =  {[P_{x} (k  +  1) V_{x} (k  +  1) P_{y} (k  +  1) V_{y} (k  +  1) ω]}^{T}, \\ F (x (k)) = [\begin{bmatrix} P_{x} (k)  +  \frac{\sin ω T}{ω} \cdot V_{x} (k)  -  \frac{1 - \cos ω T}{ω} \cdot V_{y} (k) \\ \cos ω T \cdot V_{x} (k) - \sin ω T \cdot V_{y} (k) \\ P_{y} (k)  +  \frac{1 - \cos ω T}{ω} \cdot V_{x} (k)  +  \frac{\sin ω T}{ω} \cdot V_{y} (k) \\ \sin ω T \cdot V_{x} (k) + \cos ω T \cdot V_{y} (k) \\ ω \end{bmatrix}], \\ G_{2} (K) = {[\begin{bmatrix} \frac{1}{2} T^{2} & T & 0 & 0 & 0 \\ 0 & 0 & \frac{1}{2} T^{2} & T & 0 \\ 0 & 0 & 0 & 0 & T \end{bmatrix}]}^{T}, \end{matrix}

(1)

where ω is the assumed unknown constant turn rate and

v (k)

is the process noise. Although the actual turn rate is not exactly a constant, we can assume that it is not changed in a very short-time interval. For convenience, we assume that v is a zero mean Gaussian white noise with variance

Q (k)

2.2. System Observation Model

In order to build up the estimation scheme using UKF, the sensor observation model is needed. If sensor j is used, $Z_{j} (k)$ is applied to denote the kth measurement of the target at time step $t_{k}$ . The measurement model is given by

\begin{matrix} Z_{j} (k) = h_{j} (x (k)) + v_{j} (k), \end{matrix}

(2)

where

h_{j}

is a (generally nonlinear) measurement function depending on sensor j's measurement characteristic and parameters (e.g., its location).

v_{j} (k)

is a variable representing measurement noise in sensor j. It is independent and assumed to be zero-mean Gaussian distribution white noise. The covariance of

v_{j} (k)

R_{j} (k)

3. UKF Filter-Based Human Tracking

Based on the above coordinated constant turn model and the system observation model, the unscented Kalman filter is applied to estimate the system state variable which includes the target's position coordinate and velocity.

Given the estimate $\hat{x} (k | k)$ of $x (k)$ and its estimation error covariance $P (k | k)$ , in order to avoid the linearization involved in the EKF, the UKF works by generating a set of points whose sample mean and sample covariance are $\hat{x} (k | k)$ and covariance $P (k | k)$ , respectively. The nonlinear function is applied to each of these points in turn to yield a transformed sample, and the predicted mean and covariance are calculated from the transformed samples. The samples are deterministically chosen so that they capture specific information about the Gaussian distribution.

For highly nonlinear systems, the UKF has advantages over the EKF. It avoids the linearization that causes substantial errors in the EKF for nonlinear systems and possible singular points in Jacobian matrices. The basic UKF algorithm (one cycle) can be seen in [12]. The following is the details of UKF.

3.1. Form Weighted Samples

The n-dimensional random variable $x (k)$ with mean $\hat{x} (k | k)$ and covariance $P (k | k)$ is approximated by $2 n + 1$ weighted samples or sigma points selected by the algorithm

\begin{matrix} χ_{0} (k | k) = \hat{x} (k | k), \\ W_{0} = \frac{κ}{(n + κ)}, \\ χ_{i} (k | k) = \hat{x} (k | k) + {(\sqrt{(n + k) P (k | k)})}_{i}, \\ W_{i} = \frac{1}{{2 (n + κ)}}, \\ χ_{i + n} (k | k) = \hat{x} (k | k) - {(\sqrt{(n + k) P (k | k)})}_{i}, \\ W_{i + n} = \frac{1}{{2 (n + κ)}}, \end{matrix}

(3)

where

κ \in R

{(\sqrt{(n + k) P (k | k)})}_{i}

is the ith row or column of the matrix square root of

(n + k) P (k | k)

, and

W_{i}

is the weight that is associated with the ith point. In theory, κ can be any number (positive or negative) providing that

(n + κ) \neq 0

3.2. Prediction

Given the set of samples generated by (3), the prediction procedure is as follows. (a)

Each sigma point is instantiated through the process model to yield a set of transformed samples

\begin{matrix} χ_{i} (k + 1 | k) = f_{2} [χ_{i} (k | k)] . \end{matrix}

(4)

(b)

The predicted mean is computed as

\begin{matrix} \hat{x} (k + 1 | k) = \sum_{i = 0}^{2 n} ‍ W_{i} χ_{i} (k + 1 | k) . \end{matrix}

(5)

(c)

The predicted covariance is computed as

\begin{array}{l} P (k + 1 | k) = \sum_{i = 0}^{2 n} ‍ W_{i} {χ_{i} (k + 1 | k) - \hat{x} (k + 1 | k)} \\ {\times {χ_{i} (k + 1 | k) - \hat{x} (k + 1 | k)}}^{T} . \end{array}

(6)

It is also clear that the predicted measurement is simply:

\begin{matrix} \hat{z} (k + 1) = H_{2} \hat{x} (k + 1 | k) . \end{matrix}

(7)

The difference between the measurement and the predicted observation, named the innovation, can be written as

\begin{matrix} ν (k + 1) = z (k + 1) - H_{2} \hat{x} (k + 1 | k) . \end{matrix}

(8)

The covariance of this quantity is

\begin{matrix} s_{ν} (k + 1 | k) = H_{2} P (k + 1 | k) H_{2}^{T} + σ_{r}^{2} . \end{matrix}

(9)

3.3. Calculate the Kalman Filter Gain

Use the following equation

\begin{matrix} K (k + 1) = P (k + 1 | k) H_{2}^{T} s_{ν}^{- 1} (k + 1 | k) . \end{matrix}

(10)

3.4. Update

We update the estimation using the following equations:

\begin{matrix} \hat{x} (k + 1 | k + 1) = \hat{x} (k + 1 | k) + K (k + 1) ν (k + 1), \\ P (k  +  1 | k  +  1)  =  P (k  +  1 | k)  -  K (k  +  1) s_{ν} (k  +  1) K^{T} (k  +  1) . \end{matrix}

(11)

4. Sensor Node Selection

In this section, the sensor selection method under the UKF filter will be presented. It is assumed that each sensor is able to detect the target and determine its range, and the locations of all the sensors are known. One of the approaches simply selects the nodes closest to the predicted target location as estimated by the tracker [13]. The drawback of the “closest” node approach is that it only roughly selects the sensor nodes and does not consider its contribution to the tracking accuracy and the energy consumption quantitatively and simultaneously. In this paper, we propose an adaptive sensor selection scheme similar to [14] but under UKF filter framework. It jointly selects the next tasking sensor and determines the sampling interval at the same time based on both of the prediction of the tracking accuracy and tracking energy cost.

4.1. Tracking Accuracy

Various measures can be defined based on the state estimation to stand for the tracking accuracy, such as the trace and the determinant of the covariance matrix, Fisher information defined on the Fisher information matrix which is the inverse of the state estimation covariance, eigenvalues of the difference between the desired and the predicted covariance matrix of the state, and entropy of the state estimation distribution. In this paper, based on the constant velocity model and the angular coordinated turn model, the tracking accuracy is reflected by tracking error $ϕ (k)$ at time step k which is defined as the trace of the covariance matrix $P (k | k)$ , that is,

\begin{matrix} ϕ (k) = trace (P (k | k)) . \end{matrix}

(12)

Given a predefined threshold $ϕ_{0} (k)$ , the tracking accuracy at time step k is considered to be satisfactory if

\begin{matrix} ϕ (k) < ϕ_{0} (k), \end{matrix}

(13)

otherwise it is considered to be unsatisfactory.

4.2. Energy Model

Energy consumption is used as the tracking cost. We consider the following energy model. If current sensor i selects sensor j as the next tasking sensor, then the total energy consumed by sensor i in transmission is

\begin{matrix} E_{t} (i, j) = (e_{t} + e_{d} r_{i j}^{α}) b_{c}, \end{matrix}

(14)

where

e_{t}

and

e_{d}

are decided by the specifications of the transceivers used by the nodes,

r_{i j}

is the distance between sensor i and sensor j,

b_{c}

is the number of bits sent, and α depends on the channel characteristics and is assumed to be time invariant. Energy consumed in receiving is

\begin{matrix} E_{r} (j) = e_{r} b_{c}, \end{matrix}

(15)

where

e_{r}

is decided by the specification of the receiver of sensor j. The energy spent in sensing/processing data of

b_{s}

bits by sensor j is

\begin{matrix} E_{s} (j) = e_{s} b_{s} . \end{matrix}

(16)

Therefore, the total energy consumption is

\begin{matrix} E (i, j) = E_{t} (i, j) + E_{r} (j) + E_{s} (j) . \end{matrix}

(17)

4.3. Adaptive Sensor Selection Scheme

Suppose that the current time step is k and the current tasking sensor is the sensor i which receives state estimation $\hat{x} (k - 1 | k - 1)$ and estimation covariance matrix $P (k - 1 | k - 1)$ of the time step $k - 1$ from its parent tasking sensor. It first updates the state estimation by incorporating its new measurement $Z_{j} (k)$ using the UKF algorithm described in Section 2. Then, it uses the sensor scheduling algorithm to select the next tasking sensor j and the next sampling interval $Δ t_{k}$ such that the sensor j can undertake the sensing task at the time $t_{k + 1} = t_{k} + Δ t_{k}$ . We suppose that $Δ t_{k}$ should be in the range $[T_{\min}, T_{\max}]$ , where $T_{\min}$ and $T_{\max}$ are the minimal and maximal sampling intervals, respectively. If sensor j is selected with the sampling interval $Δ t_{k}$ , its associated predicted objective function is defined as

\begin{matrix} J (j, Δ t_{k}) = w Φ_{j} (k) + (1 - w) \frac{E (i, j)}{Δ t_{k}}, \end{matrix}

(18)

where

Φ_{j} (k)

is the predicted tracking accuracy according to the UKF algorithm,

E (i, j)

is the corresponding predicted cost given by (17), the averaged energy consumption over the period.

w \in [0,1]

is the weighting parameter used to balance the tracking accuracy and the energy consumption.

The sensors are scheduled in the following two tracking methods.

(1) After prediction, none of the sensors can achieve the satisfactory tracking accuracy using any sampling interval in $T_{\min}$ and $T_{\max}$ . In this case, $Δ t_{k}$ is set to the minimal sampling interval $T_{\min}$ , and the sensor is selected by

\begin{matrix} j^{*} = \underset{j \subset A}{\arg} \min {J (j, T_{\min})}, \end{matrix}

(19)

where A is the candidate sensors that can be selected by sensor i. Generally in (19),

w \neq 0

. The purpose of this mode is to drive the tracking accuracy to be satisfactory as soon as possible with consideration of the energy consumption.

(2) After prediction, at least one sensor can achieve the satisfactory tracking accuracy. In this case, the optimal $(j^{*}, Δ t_{k}^{*})$ is selected by

\begin{matrix} (j^{*}, Δ t_{k}^{*}) = \underset{j \subset A^{*}, Φ (j, k) \leq Φ_{0}}{\arg} \min {\frac{E (i, j)}{Δ t_{k}}}, \end{matrix}

(20)

where

A^{*}

is the set of sensors that can achieve the satisfactory tracking accuracy. Equation (20) utilizes the objective function in (18) with

w = 0

. The basic idea of this mode is that when the predicted tracking accuracy is satisfactory, the sensors and the sampling interval are selected according to the energy efficiency.

For simplification, we suppose that the sampling interval is selected from predefined N values ${T_{t}}_{1}^{N}$ , where $T_{1} = T_{\min}$ , $T_{N} = T_{\max}$ , and $T_{t_{1}} < T_{t_{2}}$ if $t_{1} < t_{2}$ . In addition, the set ${T_{t}}_{1}^{N}$ is selected such that its values can evenly divide the interval $[T_{\min}, T_{\max}]$ into $N - 1$ subintervals.

5. Experimental Results

Our testbed is shown in Figure 1. All the hardwares in the testbed are supplied by Crossbow Technology. The testbed consists of the following hardwares: MicaZ (processor with on-board ZigBee radio), MDA100CA, MIB510 (USB programmer), and SRF02 (active ultrasonic sensor with I2C bus).

Figure 1

The testbed.

Figure 2 shows the MicaZ mote, which operates from the 2400 MHz to 2483.5 MHz band, and uses the Chipcon CC2420, IEEE 802.15.4 compliant, and ZigBee ready radio frequency transceiver integrated with an Atmega128L microcontroller. It has an integrated radio communication transceiver working at 2.4 GHz frequency with a transmission data rate of 250 Kbps and indoor transmission range of 20 to 30 meters. It runs TinyOS and is programmed on nesC.

Figure 2

The MicaZ mote.

The MDA100CA series sensor boards have a precision thermistor, a light sensor/photocell, and from general prototyping area. The prototyping area supports connection to all eight channels of the mote's analog to digital converter (ADC 0 to 7), both USART serial ports, and the I2C digital communications bus. The prototyping area also has 45 unconnected holes that are used for breadboard of circuitry. See Figure 3.

Figure 3

The MDA100CA sensor board.

The MIB510 interface board (see Figure 4) is a multipurpose interface board used with the MicaZ. It supplies power to the devices through an external power adapter option, and provides an interface for a RS-232 mote serial port and reprogramming port. The MIB510 has an on-board in-system processor (ISP) to program the motes. Code is downloaded to the ISP through the RS-232 serial port. The ISP programs the code into the mote. The ISP and the mote share the same serial port. The ISP runs at a fixed baud rate of 115.2 kbaud. The ISP continually monitors incoming serial packets for a special multibyte pattern. Once this pattern is detected, it disables the mote's serial RX and TX (two legs), then takes control of the serial port.

Figure 4

MIB510 programmer board.

The SRF02 (see Figure 5) is a single transducer ultrasonic range sensor. It features both I2C and a serial interfaces. I2C interface is used in this project. We use only 8 sensors in the testbed. New commands in the SRF02 include the ability to send an ultrasonic burst on its own without a reception cycle and the ability to perform a reception cycle without the preceding burst. SRF02's minimum measurement range is around 15 cm (6 inches). This sensor has a detection angle of 15 degrees and a maximum range of 6 m.

Figure 5

The SRF02 ultrasonic sensor.

The developed target tracking system, see Figures 1 and 6, is made up of 8 ultrasonic sensor nodes. These 8 ultrasonic sensors arelocated along the edge of the area, respectively, with coordinates (200, 0), (250, 170), (50, 300), (0, 110), (100, 0), (250, 60), (150, 300), and (0, 230). The orientations of the sensors (clockwise from the positive x-axis) are, respectively, 65°, 90°, 50°, 75°, 100°, 110°, 90°, and 120° such that the sound waves are not reflected from nearby walls/obstacles. Each node is allocated with an ID number and an XY coordinate. Their locations are shown in Figure 1 to cover a monitoring area of 2.5 m × 3.0 m. The tracking target is a human. A MicaZ mote will be attached to each sensor node.

Figure 6

The human tracking system.

On the base station, a laptop is connected to the network through a MicaZ mote for receiving data packets via USB connection. Figure 6 shows the tracking system deployed in the testbed. Upon receiving an initial time synchronizing beacon from processing mote, all sensor nodes will initialize their starting time for sensor nodes. These sensor nodes will broadcast their sensor readings with one sensor reading at a time to the processing mote to avoid sensors' interference. The processing mote will also program the default measurement for each sensor.

The real-time data is collected from a human who is moving around within the sensor coverage area of the testbed. The sensor selection scheme is performed during the data collection process. In order to simplify the sensor selection algorithm, we selected one sensor at each time step in the experiments. The data collected is run by UKF filter-based tracking approach proposed in Section 2, and we compared the experimental results with the method in [13]. Figure 7 to Figure 10 shows the comparison of the result with UKF and EKF, respectively. We can see that the result from UKF is better than the results from EKF in the real testbed.

Figure 7

The human tracking system experimental result with UKF and EKF.

Figure 8

The human tracking error comparison in X direction of the testbed with UKF and EKF.

Figure 9

The human tracking error comparison in Y direction of the testbed with UKF and EKF.

Figure 10

The human tracking error comparison of the MSE (mean square root error) of the testbed with UKF and EKF.

6. Conclusions

This paper presents a UKF filter-based adaptive sensor scheduling scheme for human tracking in wireless sensor networks. It uses cheap range sensor nodes in wireless sensor networks by jointly selecting the next tasking sensor and determining the sampling interval based on predicted tracking accuracy and tracking cost under the UKF filter frame. Simulation results show that the new scheme can achieve significant energy efficiency without degrading the tracking accuracy. There are still many issues remaining for future study. Multistep, multisensor selection based adaptive sensor scheduling and sensor scheduling for multitarget tracking are both challenging problems for further investigations.

References

Mittal

Davis

L. S.

M2Tracker: A multi-view approach to segmenting and tracking people in a cluttered scene using region-based stereo

Proceedings of European Conference on Computer Vision

June 2002

Copenhagen, Denmark

18 36

Cai

Aggarwal

J. K.

Tracking human motion in structured environments using a distributed-camera system

IEEE Transactions on Pattern Analysis and Machine Intelligence 1999 21 11 1241 1247

2-s2.0-0033220872

Zajdel

Cemgil

A. T.

Brose

B. J. A.

Dynamic bayesian networks for visual surveillance with distributed cameras

Proceedings of the 1st European Conference on Smart Sensing and Context

October 2006

Enschede, Netherlands

240 243

Tan

Wang

Maybank

A survey on visual surveillance of object motion and behaviors

IEEE Transactions on Systems, Man and Cybernetics C 2004 34 3 334 352

2-s2.0-3543105519

10.1109/TSMCC.2004.829274

Mallick

La Scala

B. F.

IMM estimator for ground target tracking with variable measurement sampling intervals

Proceedings of the 9th International Conference on Information Fusion (FUSION '06)

July 2006

Florence, Italy

1 8

2-s2.0-50149121578

10.1109/ICIF.2006.301763

Mazor

Averbuch

Bar-Shalom

Dayan

Interacting multiple model methods in target tracking: a survey

IEEE Transactions on Aerospace and Electronic Systems 1998 34 1 103 123

2-s2.0-0031647084

Sensor management based on cross-entropy in interacting multiple model kalman filter

Proceedings of the 2004 American Control Conference (AAC)

July 2004

Boston, Mass, USA

5381 5386

2-s2.0-8744304833

Luo

Z. X.

Jannett

T. C.

Modeling sensor position uncertainty for robust target localization in wireless sensor networks

Proceedings of the IEEE Radio and Wireless Symposium

Janurary 2012

Santa Clara, Calif, USA

Luo

Z. X.

Jannett

T. C.

Energy-based target localization in multi-hop wireless sensor networks

Proceedings of the IEEE Radio and Wireless Symposium

Janurary 2012

Santa Clara, Calif, USA

10.

Luo

Z. X.

Jannett

T. C.

Multi-objective method to balance energy consumption and performance for energy-based target localization in wireless sensor networks

Proceedings of the IEEE Southeastcon

March 2012

Orlando, Fla, USA

11.

Hao

Brady

D. J.

Guenther

B. D.

Burchett

J. B.

Shankar

Feller

Human tracking with wireless distributed pyroelectric sensors

IEEE Sensors Journal 2006 6 6 1683 1695

2-s2.0-33746353472

10.1109/JSEN.2006.884562

12.

Xiong

Zhang

H. Y.

Chan

C. W.

Performance evaluation of UKF-based nonlinear filtering

Automatica 2006 42 2 261 270

2-s2.0-29244450170

10.1016/j.automatica.2005.10.004

13.

Toh

Y. K.

Xiao

Xie

A wireless sensor network target tracking system with distributed competition based sensor scheduling

Proceedings of the 3rd International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP '07)

December 2007

Melbourne, Australia

257 262

2-s2.0-51349090108

10.1109/ISSNIP.2007.4496853

14.

Xiao

W. D.

J. K.

Xie

L. H.

Dong

Sensor scheduling for target tracking in networks of active sensors

Acta Automatica Sinica 2006 32 6 922 928

2-s2.0-33846105411

A Novel Human Motion Tracking Approach Based on a Wireless Sensor Network

Abstract

1. Introduction

2. Problem Formulation

2.1. Coordinated Turn Model

2.2. System Observation Model

3. UKF Filter-Based Human Tracking

3.1. Form Weighted Samples

3.2. Prediction

3.3. Calculate the Kalman Filter Gain

3.4. Update

4. Sensor Node Selection

4.1. Tracking Accuracy

4.2. Energy Model

4.3. Adaptive Sensor Selection Scheme

5. Experimental Results

6. Conclusions

References