Sage Journals: Discover world-class research

Abstract

Traditional fall detection systems require to wear special equipment like sensors or cameras, which often brings the issues of inconvenience and privacy. In this article, we introduce a novel multistage fall detection system using the channel state information from WiFi devices. Our work is inspired by the fact that different actions have different effects on WiFi signals. By fully analyzing and exploring the channel state information characters, the falling actions can be distinguished from other movements. Considering that falling and sitting are very similar to each other, a special method is designed for distinguishing them with deep learning algorithm. Finally, the fall detection system is evaluated in a laboratory, which has 89% detection precision with false alarm rate of 8% on the average.

Keywords

Channel state information fall detection system WiFi devices

Introduction

Falls can be depicted as the abrupt change of human body from an upright position to a lying-down position without control. According to a 5-year prospective study, 668 incidents of falls per 1000 adults over the age of 65 occur annually, and the frequency increases for successive age groups above the age of 75 years.¹ Thus, it is necessary to deploy the automatic fall detection (FD) system in a home environment for reducing the rescue time. Traditional FD systems mainly adopt four types of devices, which are ambient devices, wearable sensors, smartphones, and cameras, respectively. FD systems using ambient devices leverage floor vibration caused by a fall to detect a risky situation.² Wearable sensor-based^3,4 and smartphone-based⁵ techniques employ sensors to capture the changes of the acceleration or velocity. Such systems require users to carry special devices on the body and are thus inconvenient. Camera-based systems employ activity classification algorithms⁶ of images, thereby effectively detecting a fall. However, these systems are fundamentally limited because they are affected by obstacles or lighting conditions. Moreover, cameras are expensive and may compromise personal privacy.⁷ The above limitations hinder the popularization and application of FD systems.

Recently, the pervasiveness of WiFi signals spur a surge in relevant research on motion detection,^8,9 gesture recognition,^10,11 and localization.^12,13 Many advances have been made in terms of accuracy and granularity. Wang and his colleagues^14,15 design a truly unobtrusive FD system based on WiFi, called WiFall, which employs physical layer channel state information (CSI) values from Intel 5300 network interface card by simply modifying the driver.¹⁶ It can detect human fall without requiring hardware modification, using extra environmental setup, or wearing additional devices. Unfortunately, the falling action is very similar with sitting, and thus the two actions cannot be distinguished effectively, which affects the accuracy of WiFall seriously.

To address this issue, we attempt to construct a novel WiFi-based multistage FD system called WmFall, which can accurately detect a falling action by exploring the changes of WiFi signals. For a human doing various actions, the WiFi signal reflected by the human body generates unique characters. These CSI variations allow us to employ signal processing techniques to classify the action information such as walking, falling, standing, and running. Falling and sitting have a similar effect on WiFi signals. However, we argue that the CSI values obtained after these two actions are different. Based on the above analysis, FD can be divided into two stages:¹ initial judgment stage, wherein falling and sitting are discriminated from other human activities;² final judgment phase, wherein the action of sitting is distinguished from falling. Specifically, our main contributions are as follows:

We design a novel CSI-based device-free FD system with deep learning. Different from the previous works, the proposed method can effectively distinguish the falling actions from sitting.

Extensive experiments have been conducted in the laboratory environment under various situations. Compared with the typical systems, our system achieves a better performance.

The rest of the article is organized as follows: we introduce the CSI and present the preliminaries in section “Preliminary.” Section “System design” introduces the main system design. The system performance is evaluated in section “Experiments.” Finally, we conclude the whole work in section “Conclusion.”

Preliminary

CSI

Modern WiFi devices that meet IEEE 802.11n/ac standard typically support multiple-input multiple-output (MIMO). These WiFi devices continuously monitor the wireless channel state to effectively perform power allocations and rate adaptations for each individual MIMO stream such that the available capacity of the wireless channel is maximally utilized.¹⁷ The states of the channel can be quantified in terms of CSI values, which essentially characterize the channel frequency response (CFR) for each subcarrier.^18,19 In a word, CSI estimates the channel properties of a communication link and describes how a signal propagates in the channel.

The effectiveness of CSI

Figure 1 shows the obtained received signal strength indication (RSSI) and CSI values during the stand-fall-after falling experiment. The results in Figure 1(a) clearly show that RSSI varies for the same action over the time. And there is no clear correlation between RSSI and activities. Different from RSSI, the CSI shows favorable correlations from Figure 1(b). Through the experiments, CSI is confirmed to be more informative than RSSI²⁰ and thus more sensitive to FD.

Figure 1.

The influence on (a) RSSI and (b) CSI.

The ambiguity of CSI

We first refine the original CSI matrix by singular value decomposition (SVD) and then arrange the amplitude information in chronological order to obtain the matrix sequence. After SVD, singular values are arranged in descending order of diagonal matrix. Each singular value can show the characteristics of the action.¹⁴ As shown in Figure 2, a singular value curve can be used for distinguishing the characteristics of different actions. However, the curves of falling and sitting are very similar to each other. Namely, direct classification of singular values may not be able to identify falling and sitting. In order to solve the problem of similarity between them, we further judge the subsequent amplitude of the suspected fall.

Figure 2.

Eigenvalues of SVD matrix.

Empirical data as motivation

The singular value curves of falling and sitting are similar because there is a certain degree of similarity in speed and motion between them. However, we notice another phenomenon, that is, the two actions are stabilized at different altitudes. As shown in Figure 3(a), the amplitude values of follow-ups after the falling and sitting are different. Therefore, it inspires us to further study the average amplitude of the follow-up action. Figure 3(b) presents the average amplitude of each subcarrier. We can see that their average values are obviously different. Thus, the follow-up signals of the falling and sitting can be used to classify the two actions.

Figure 3.

The CSI amplitude: (a) the amplitude of follow-ups and (b)average amplitude comparison.

System design

In this section, we give the whole system design. And each part of the system will be described one by one.

System overview

Our system utilizes commercial WiFi devices to detect human fall by recognizing the different activities of people. Figure 4 shows the whole system architecture, which are divided into three components: sensing, initial judgment, and final judgment.

Figure 4.

WmFall system architecture.

In the first part, the access points are used as the transmitters, and the receivers are existing devices at home (e.g. laptop computer), which need to be equipped with Intel WiFi 5300. Raw CSI values are collected from a chip set firmware to receive packets.

For the second part, various data preprocessing methods are applied to the raw CSI stream for the minimization of interference from environment noise. Next, features can be extracted from the CSI data, which are used for training a classification model based on the support vector machine (SVM).^21,22 The obtained classification model is then employed to recognize different activities, which can get an initial judgment of fall. The initial judgment can distinguish most of the movements, except for sitting and falling. The main purpose of the third part is to distinguish the two movements.

The third part serves as a final determination for distinguishing falling from sitting, which includes an offline training phase and an online recognition phase. In the offline training stage, deep learning algorithm^23,24 is utilized to train a classification model for distinguishing sitting and falling using CSI data, which have been collected after sitting and falling. In online recognition stage, the trained classification model is adopted to recognize the above two actions, in order to make a final judgment of FD.

Data preprocessing

In this section, we introduce two data preprocessing methods to deal with the raw noisy CSI data, for the purpose of improving the robustness and accuracy of the FD system. Low-pass filtering is first leveraged to filter out high-frequency ambient noise. Furthermore, we introduce the discrete wavelet denoising to distinguish mutant from partial noise. CSI data can be extracted from WiFi data packet, each CSI is a three-dimensional matrix of $N_{t} \times N_{r} \times N_{c}$ , where $N_{t}$ is the number of antennas at the transmitting end, $N_{r}$ is the number of antennas at the receiving terminal, and $N_{c}$ is the number of subcarriers. In our experiment, we set $N_{t}$ to 1, $N_{r}$ to 3, in the orthogonal frequency–division multiplexing (OFDM) channel, $N_{c}$ is 30. Thus, each CSI packet is divided into three streams, and each stream contains 30 subcarriers, which can be expressed as

\begin{array}{l} C S I^{1} = {C S I^{1,1}, C S I^{1,2}, \dots, C S I^{1,30}} \\ C S I^{2} = {C S I^{2,1}, C S I^{2,2}, \dots, C S I^{2,30}} \\ C S I^{3} = {C S I^{3,1}, C S I^{3,2}, \dots, C S I^{3,30}} \end{array}

In WmFall, the Intel WiFi 5300 includes three antennas, each of which can collect CSI data from 30 different subcarriers. In all, 90 raw CSI measurements are obtained for each packet reception. We only consider the amplitude responses in this article since the phases of CSI are often instable.

Raw CSI data have some high-frequency band noise due to hardware problems. To mitigate the effect of these noise data, we select the Butterworth low-pass filter.²⁵ The characteristic of Butterworth low-pass filter is that the frequency-response curve in the pass band is flattened without any ups and downs. In addition, the resistance band is gradually reduced to zero so that the signal affected by a specific action has a high degree of fidelity and the outliers of the noise signal can be reduced as much as possible. In the experimental observation, we find that setting the cutoff frequency to 60 Hz retains the action information better and achieves a better filtering effect. Figure 5 shows the raw CSI data and the filtered data by Butterworth low-pass filter for a single subcarrier. The results show that low-pass filter can effectively remove the high-frequency band noise data.

Figure 5.

Data preprocessing.

The Butterworth low-pass filter eliminates most of the high-frequency noise that cannot be reliably differentiated from mutant motion signals and partial noise. Smooth noises result in the blurring of signals. Thus, CSI streams must be processed by a discrete wavelet denoising method.²⁶ Discrete wavelet denoising has a high-frequency resolution and a low-temporal resolution for low-frequency parts. The high-frequency part has a high time resolution and a low-frequency resolution. Thus, it is suitable for the detection of normal signals in mutated signal components. This part ensures that the resolution of the processed signal remains high. Figure 5 shows the CSI data after discrete wavelet denoising.

Initial judgment

After processing the raw CSI data, the system will determine whether a action has occurred. In order to provide a reliable basis for the action classification, we perform SVD on the CSI matrix so that the detection mechanism scans the singular values of different behaviors. SVD can transform correlated variables into a set of uncorrelated variables, which better exposes the various relationships among the original data. It can represent a complex matrix with a small and simple multiplication of several sub-matrices. These small matrices can describe the important properties of the matrix. It is also a method for best approximation of the original data points using fewer dimensions. The basis of eigenvector decomposition is spectral analysis. SVD is the generalization of the spectral analysis theory on any matrix. Therefore, the singular value has a very wide application in the field of statistics and signal processing. In the proposed system, the receiver receives 100 CSI packets per second. In order to show the characteristics of all the actions, we set the window value to 10. Usually, the signal state in the ideal experimental environment tends to be stable so that each sample is a low-latitude matrix. We find that each sample in the SVD matrix can represent the characteristics of the entire matrix. The sample values are decremented in terms of the size of the matrix. In order to facilitate the intuitive observation of the data changes, we select the second eigenvalue for classification. The experimental and classification results show that there is a distinction between each action. Using singular values can classify behaviors more quickly and intuitively. With the features extracted from profile construction module, SVM is used for FD. The classification of the obtained singular value algorithm allows us to initially determine whether there has been a suspected fall action.

Final judgment

After the initial judgment, we further design a fine-grained judgment method to distinguish the falling and sitting actions. Through the preliminary experiment, we find the average CSI values are different after falling and sitting. Based on this observations, we propose a method to make a final judgment of fall with CSI data. First, we collect the training data after a user falls and sits near the sofa and other seats, to feed the classification algorithm. With these data, the deep learning algorithm is used to obtain a classification model for final fall judgment, which contains three stages: pre-training, unrolling, and fine tuning as shown in Figure 6. Here, a deep network with three hidden layers is adapted, where every hidden layer consists of different number of neurons.

Figure 6.

Weight training with deep learning.

To reduce the dimension of CSI data, we assume that the number of neurons in a previous hidden layer is more than that of latter hidden layer. Let $K_{1}$ , $K_{2}$ , and $K_{3}$ denote the number of neurons in the first, second, and third hidden layers, respectively. Thus, $K_{1} > K_{2} > K_{3}$ .

In addition, we propose a new approach to represent CSI values. We use the weights between connected layers, with $W^{(1)}$ , $W^{(2)}$ , $W^{(3)}$ defined as the weights between the normalized magnitudes of CSI values and the first hidden layer, the first and second hidden layers, and the second and third hidden layers, respectively. The key idea is that, after training the weights in the deep network, we can store them as CSI values to facilitate gesture recognition in the test stage. Here, $J (w, b)$ is used as the loss function to calculate the weights, which is defined as follows

J (w, b) = - ℤ_{x, y \sim \hat{p} d a t a} \log p_{m o d e l} (y | x)

(1)

We use neuro-linguistic programming (NLP) model to connect the deep network and denote rectified linear unit (RELU) function as the activation function. The output of each layer can be represented as follows

{\begin{matrix} h^{(1)} = g^{(1)} ({W^{(1)}}^{T} x + b^{(1)}) \\ h^{(2)} = g^{(2)} ({W^{(2)}}^{T} x + b^{(2)}) \\ h^{(3)} = g^{(3)} ({W^{(3)}}^{T} x + b^{(3)}) \end{matrix}

(2)

where $h^{(i)}$ is the output of the ith layer, $g^{(i)} ()$ is the corresponding activation function, and $b (i)$ is the bias.

To calculate the weights of each layer, the gradient decent algorithm is adopted

(w, b) = (w, b) - η \cdot \nabla_{(w, b)} J ((w, b); x_{i : i + m}; y_{i : i + m})

(3)

Complexity analysis: the main time complexity of this algorithm is in SVM and deep neural network, the complexity of the training process is high, but we only need to use a trained model to identify. The test complexity of these two algorithms is low and it only has one order of magnitude, which ensures the effectiveness of this algorithm. Only the complexity of SVD needs to be considered. Suppose the matrix A is $N \times M$ , then perform SVD. The complexity is $O (N^{3})$ for each execution.

Experiments

This section describes the experimental setup of the whole system and gives the detection results.

Experimental setting

As shown in Figure 7, we employ a commercial TP-LINK TL-WR741N wireless router as the transmitter, which works with 2.4G frequency band and IEEE 802.11n-based protocol. The TL-WR741N router transmits 100 packets per second, while a computer equipped with Intel 5300 wireless card is adopted as a receiver to extract CSI measurements. The two devices are deployed in a $6 m \times 8 m$ laboratory. Two metrics are defined to evaluate the performance in this article:

True positive (TP) rate is the ratio that human FD is correctly identified, which is calculated as follows

TP = \frac{number of truly detected falls}{number of total falls}

False positive (FP) rate is the ratio that the system generates an alarm when there is no fall, which is denoted as

FP = \frac{number of mistakenly detected falls}{number of total falls}

In the experiment, six volunteers are asked to perform different actions such as standing, walking, sitting down, falling, and other actions. Each volunteer does 20 times for each action and CSI data are collected at the same time. In addition, to test the performance of final judgment, we collect 10 sets of follow-up amplitude data of sitting down and falling down for each volunteer.

Figure 7.

Experiment scenario.

Initial judgment performance

To prove that human movements can affect the wireless signal propagation, we collected CSI values when volunteers perform four different series of actions (stand → fall → after-fall, stand → sit → after-sit, walk → fall → after-fall, walk → sit → after-sit). Thereafter, we analyze these CSI data by computing the second largest eigenvalue from the SVD. Figure 8 shows the change variance of each group of second largest eigenvalue when humans do different activities. When a person stands and walks, the eigenvalue is extremely low. Nevertheless, both sitting and falling cause the great variance. Thus, falling and sitting can be clearly distinguished from other human activities like standing and walking.

Figure 8.

Variance of the second largest eigenvalue: (a) stand–fall, (b) stand–sit, (c) walk–fall, and (d) walk–sit.

We first evaluate the detection performance of the initial judgment, in which the activity sets consist of falling, sitting, standing, and walking. Also, the effects of the number of antennas on the initial judgment are studied. It is easy to see that more antennas bring higher precision. Figure 9(a) and (b) shows the TP and FP rate of initial judgment, respectively. For standing and walking, in the initial judgment stage high TP rate and low FP rate can be achieved which has a mean of more than 95% and less than 11%. For falling and sitting, the initial judgment has a poor performance with TP rate of only 83% on the average especially that the FP rate is more than 31%. Thus, this initial judgment has a good performance for distinguishing falling and sitting from other human activities. However, it is difficult to distinguish fall and sit based on this initial judgment only.

Figure 9.

Performance comparison: (a) TP and (b) FP of initial judgment and (c) performance of NLOS.

We also evaluate the performance of WmFall in non-line-of-sight (NLOS) locations. In addition, we use a desk to block the line-of-sight (LOS) path of the transmitter–receiver link. Then we ask volunteers to conduct activities at NLOS locations. Figure 9(c) shows that the FD rate decreases to around 65%. The detection rate of other activities also decreases.

Final judgment performance

This subsection presents the performance of the final judgment. A volunteer is asked to perform two different activities (falling on the ground near the chair and sitting on the chair). Here, 1000 CSI data packets under the influence of two activities are used to train a classification with deep neural network. The volunteer consecutively performs the two actions 100 times for high confidence. Figure 10 shows a good performance with 91% accuracy when using three antennas.

Figure 10.

Final judgment performance.

Performance comparison

We compare the performance between WiFall and WmFall in different distances, which is from 1 to 3.5 m. As shown in Figure 11, we can find that WmFall not only has a higher TP but also has a lower FP than WiFall. In particular, Figure 11(b) shows that the FP of the WmFall is at almost 0. Thus, WmFall has a better performance than WiFall, which has 89% detection precision with false alarm rate of 8% on the average.

Figure 11.

Comparison between WmFall and WiFall in (a) TP and (b) FP.

Conclusion

In this article, we introduce a novel CSI-based multistage FD scheme with deep learning, which consists of three processes: sensing, initial judgment, and final judgment. In the sensing part, raw CSI data are collected by utilizing access points. In the initial judgment, low-pass filter and discrete wavelet denoising are applied for processing raw CSI data. A classification model is then trained to recognize different activities. This part can distinguish most of the movements, except the actions of sitting and falling. The final judgment includes an offline training phase and online recognition phase, which can distinguish falling from sitting. Extensive experiments have confirmed the effectiveness of the proposed system.

Footnotes

Handling Editor: Yu Wang

Authors’ Note

Xu Yang, Yuan Shao and Qiang Niu are also affiliated with Mine Digitization Engineering Research Center of the Ministry of Education, Xuzhou, China.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported, in part, by the Fundamental Research Funds for the Central Universities (Grant No. 2017XKQY080).

ORCID iD

Xu Yang

References

Alwan

Rajendran

Kell

et al . A smart and passive floor-vibration based fall detector for elderly. In: Proceedings of the 2nd international conference on information & communication technologies, 24–28 April 2006, Damascus, Syria, pp.1003–1007. New York: IEEE.

Noury

Fleury

Rumeau

et al . Fall detection: principles and methods. In: Proceedings of the 29th annual international conference of the IEEE engineering in medicine and biology society, Lyon, 22–26 August 2007, pp.1663–1666. New York: IEEE.

Zhao

Zhong

et al . Development of a wearable-sensor-based fall detection system. Int J Telemed Appl2015; 2015: 2.

Rimminen

Lindström

Linnavuo

et al . Detection of falls among the elderly by a floor sensor using the electric near field. IEEE Trans Inf Technol Biomed2010; 14(6): 1475–1476.

Abbate

Avvenuti

Bonatesta

et al . A smartphone-based fall detection system. Pervasive Mob Comput2012; 8(6): 883–899.

Cucchiara

Prati

Vezzani

A multi-camera vision system for fall detection and alarm generation. Expert Syst2010; 24(5): 334–345.

Kepski

Kwolek

Fall detection using ceiling-mounted 3D depth camera. In: Proceedings of the international conference on computer vision theory and applications, Lisbon, 5–8 June 2014, pp.640–647. New York: IEEE.

Qian

Yang

et al . Decimeter level passive tracking with WiFi. In: Proceedings of the 3rd workshop on hot topics in wireless, 3–7 October 2016, New York, pp.44–48. New York: ACM.

Yang

Zhou

et al . Non-invasive detection of moving and stationary human with WiFi. IEEE J Sel Area Comm2015; 33(11): 2329–2342.

10.

Yang

Wang

et al . Wifinger: talk to your smart devices with finger-grained gesture. In: Proceedings of the ACM international joint conference on pervasive and ubiquitous computing, Heidelberg, 12–16 September 2016, pp.250–261. New York: ACM.

11.

Tan

Sheng . Wifinger: leveraging commodity WiFi for fine-grained finger gesture recognition. In: Proceedings of the 17th ACM international symposium on mobile ad hoc networking and computing, Paderborn, 4–8 July 2016, pp.201–210. New York: ACM.

12.

Wang

Gao

Mao

CSI phase fingerprinting for indoor localization with a deep learning approach. IEEE Internet Things2017; 3(6): 1113–1123.

13.

Wang

Gao

Mao

et al . CSI-based fingerprinting for indoor localization: a deep learning approach. IEEE T Veh Technol2017; 66(1): 763–776.

14.

Wang

LM.

Wifall: device-free fall detection by wireless networks. IEEE T Mobile Comput2017; 16(2): 581–594.

15.

Han

Wang

et al . Wifall: device-free fall detection by wireless networks. In: Proceedings of the IEEE conference on computer communications, Toronto, Ontario, 27 April–2 May, pp.271–279. New York: IEEE.

16.

Halperin

Sheth

et al . Predictable 802.11 packet delivery from wireless channel measurements. In: Proceedings of the ACM SIGCOMM 2010 conference, New Delhi, India, 3 September 2010, New Delhi, India, pp.159–170. New York: ACM.

17.

Zheng

Wang

Shangguan

et al . Smokey: ubiquitous smoking detection with commercial wifi infrastructures. In: Proceedings of the IEEE international conference on computer communications, San Francisco, CA, 10–14 April 2016, pp.1–9. New York: IEEE.

18.

Yang

Zhou

Liu

From RSSI to CSI: indoor localization via channel response. ACM Comput Surv2013; 46(2): 1–32.

19.

Xiao

et al . FILA: fine-grained indoor localization. IEEE Infocom Ser2012; 131(5): 2210–2218.

20.

Zhou

Yang

et al . Sensorless sensing with WiFi. Tsinghua Sci Technol2015; 20(1): 1–6.

21.

Ding

Zhang

et al . Weighted linear loss multiple birth support vector machine based on information granulation for multi-class classification. Pattern Recogn2017; 67: 32–46.

22.

Xiao

Wang

Parameter selection of Gaussian kernel for one-class SVM. IEEE T Cybernetics2017; 45(5): 941–953.

23.

Zhang

Ding

Zhang

et al . Research on point-wise gated deep networks. Appl Soft Comput2016; 25: 1210–1221.

24.

Schmidhuber

Deep learning in neural networks: an overview. Neural Network2014; 61: 85–117.

25.

Chen

Yang

Pan

The dynamic response of a Butterworth low-pass filter in an ac-based electrical capacitance tomography system. Meas Sci Technol2010; 21(10): 105505.

26.

Wang

YR.

Novel image denoising method based on discrete fractional orthogonal wavelet transform. Acta Electronica Sinica2014; 42(2): 280–287.

WmFall: WiFi-based multistage fall detection with channel state information

Abstract

Keywords

Introduction

Preliminary

CSI

The effectiveness of CSI

The ambiguity of CSI

Empirical data as motivation

System design

System overview

Data preprocessing

Initial judgment

Final judgment

Experiments

Experimental setting

Initial judgment performance

Final judgment performance

Performance comparison

Conclusion

Footnotes

Authors’ Note

Declaration of conflicting interests

Funding

ORCID iD

References