Sage Journals: Discover world-class research

Abstract

Physical activity (PA) recognition has recently become important in activity monitoring for the public healthcare. Although body-worn sensors are well suited to collect data on activity patterns for long periods of time, users may forget to wear special microsensors. On the contrary, more and more people take smartphone with them almost anytime. At present, most popular smartphones have three built-in kinematic sensors (triaccelerometer, gyroscope, and magnetic sensor) which could be utilized to recognize PA. This study utilized three built-in kinematic sensors in a smartphone to recognize PA and found out which features derived from the three sensors were significant to different PA. We used a combined algorithm of Fisher's discriminant ratio criterion and $J_{3}$ criterion for feature selection. A hierarchical classifiers system including fourteen classifiers was proposed and employed to recognize fifteen activities. The optimal features derived from the built-in kinematic sensors of the smartphone were selected from 140 features. The results indicated that the accelerometer was significant to PA recognition, while gyroscope and orientation sensor were effective to recognize the change of body posture and detect falls, respectively. The total classification accuracy of 95.03% demonstrated the feasibility of utilizing the built-in kinematic sensors of the smartphone to recognize PA.

1. Introduction

Physical activity (PA) recognition has recently become important in activity monitoring for the public healthcare. The progressive decline in the PA level due to the adoption of sedentary lifestyles has been associated with the increasing incidence of obesity, diabetes, and cardiovascular diseases [1, 2]. In addition, a “long lie” caused by a fall may result in negative psychological and physical outcomes [3]. The fear of falling can lead to a further decrease of physical activity. Thus, a reliable PA recognition system in daily life would not only encourage people to do more outdoor activities but also be an assessment of activities of daily living for chronic treatment.

Body-worn sensors [4, 5] are well suited to collecting data on activity patterns for long periods of time. In contrast to other approaches, such as laboratory-based systems [6] or video analysis [7, 8], they can be used under conditions of free living with minimal inconvenience to the user [9]. However, users may forget to wear these special microsensors. At present, smartphone is a fast-growing device which people take with it anytime. Most of state-of-the-art smartphones have the built-in kinematic sensors (such as triaccelerometer, gyroscope, and magnetic sensor), Global Positioning System (GPS), Wi-Fi, Bluetooth, camera, proximity sensor, microphone, and so forth.

The built-in kinematic sensors of smartphones are used to recognize PA, which not only reduces the cost of hardware but also exploits existing communication module and ubiquitous monitor. The previous studies have demonstrated the effectiveness of using a smartphone to detect a fall [10]. Acceleration was measured by two devices, a smartphone and an independent accelerometer, to confirm that fall detection using a smartphone is a feasible and highly attractive technology.

Most of the previous studies [10–12] only used accelerometers to research PA recognition. In our earlier study [13], we also utilized accelerometer built in a smartphone to detect falls and raised the alarm by Multimedia Messaging Service (MMS). However, acceleration only reflects the change of PA in linear acceleration, but angular velocity and orientation information were not reflected by an accelerometer.

There are several studies using the combination of accelerometer, gyroscope, and magnetic sensor. Zhang et al. [14] evaluated a continuous activity recognition system, using three small wearable wireless sensors and a smartphone to store, transmit, analyze, and update data. Three sensors, each composed of a triaxial accelerometer and a triaxial gyroscope, are attached to the chest and both thighs. The recognition algorithm was developed using a decision tree based on time series data and spectrum analysis. Bahle et al. [15] found a particularly interesting result that adding the magnetic disturbance features significantly improves recognition based on the vector norm of accelerometers and gyroscopes. Chiang et al. [16] measured essential movements of human body through wireless sensor network body motion sensors that involve the accelerometer and gyroscope. Activities are recognized by the fuzzy algorithm. Jeannet et al. [17] used a monitor containing a 3D accelerometer and a gyroscope fixed on the subject's tee-shirt, on the chest. Multiple sensors systems could identify physical activity in a more refined way by recognizing subactivities such as jogging or rowing, due to the fact that a bigger amount of information on the body motion can be collected. However, the more the sensors were employed to record physical activity, the higher the interference of the measuring system with the spontaneous behavior of the user [18]. Therefore, the development of simple, small and light-weight systems to monitor physical activity in daily life should be recommended.

The aims of this study were to utilize the built-in kinematic sensors (triaccelerometer, gyroscope, and magnetic sensor) of a smartphone to recognize PA and find out which features derived from the built-in kinematic sensors were significant to recognize PA. This work will make the assessment of activities of daily living and fall detection more ubiquitous and portable.

The rest of the paper is organized as follows: Section 2 provides the method of the PA recognition. The experimental results and discussion are presented in Sections 3 and 4. Finally, Section 5 concludes the paper.

2. Methods

The block diagram of the main data processing scheme was described in Figure 1. The built-in accelerometer, gyroscope, and the orientation sensor of a smartphone collected information which reflected acceleration, angular velocity, and orientation of PA. And then features were extracted from the information, and the optimal features were selected. Subsequently, all activities were classified, and if a fall was detected, the alarm information about the location of the fall would be sent to the preset person by MMS.

Figure 1

The block diagram of the main data processing scheme.

2.1. Device and Data Collection

A smartphone (Samsung I9023 Nexus S, $125 \times 63 \times 11.2$ mm, 129 g, Android operating system version 2.3.6, Samsung, Korea) was placed inside an adjustable band attached to the chest of a subject, as shown in Figure 2. The activity recognition system requires firm attachment of the smartphone to the subject.

Figure 2

The coordinate system and the placement of the smartphone at the chest of a subject.

The smartphone has built-in triaxial accelerometer (STM KR3DM) with 19.6 m/s² maximum range and 0.019 m/s² resolution, triaxial gyroscope sensor (STM K3G) with 34.9 rad/s maximum range and 0.0012 rad/s resolution, and triaxial magnetic field sensor (Asahi Kasei AK8973) with 2000 μT maximum range and 0.0625 μT resolution. An application software of PA recognition was developed and installed on the smartphone. This application measured the acceleration, angular velocity, and orientation of the smartphone.

The standard sensor coordinate system of the smartphone is defined relative to the screen. As shown in Figure 2, the x-axis ( $X_{s}$ ) is horizontal and points to the right. The y-axis ( $Y_{s}$ ) is vertical and points up, and the z-axis ( $Z_{s}$ ) points toward the outside of the screen. The $X_{s}$ , $Y_{s}$ , and $Z_{s}$ of smartphone are according to the mediolateral, vertical, and dorsoventral directions of the subject, respectively.

The coordinate system of the accelerometer is the same as the standard sensor coordinate system. The accelerometer measures acceleration along $X_{s}$ ( $A_{x}$ ), $Y_{s}$ ( $A_{y}$ ), and $Z_{s}$ ( $A_{z}$ ), respectively. The low pass filter with 0.25 Hz cutoff frequency was employed to separate acceleration force to gravity acceleration (GA) and linear acceleration (LA). So $A_{x}$ was separated into ${GA}_{x}$ and ${LA}_{x}$ , the same as $A_{y}$ and $A_{z}$ .

The coordinate system of the gyroscope is the same as the one used for the accelerometer. Rotation is positive in the counterclockwise direction. Gyroscope measures the rate of rotation around $X_{s}$ ( $G_{x}$ ), the rate of rotation around $Y_{s}$ ( $G_{y}$ ), and the rate of rotation around $Z_{s}$ ( $G_{z}$ ) in radian/s.

The orientation sensor is software-based and derives its data from the accelerometer and the geomagnetic field sensor. The orientation sensor monitors the position of the smartphone relative to the earth's frame of reference and measures azimuth ( $O_{a}$ ), pitch ( $O_{p}$ ), and roll ( $O_{r}$ ). $O_{a}$ is the angle between magnetic north of the earth and the positive $Z_{s}$ of the smartphone. When $Z_{s}$ is aligned with magnetic north, the $O_{a}$ value is 0, and when $Z_{s}$ is pointing south, the $O_{a}$ value is 180. $O_{p}$ is the angle between the horizontal plane of the earth and the positive $Z_{s}$ . $O_{p}$ is positive when the positive $Z_{s}$ rotates toward the positive $Y_{s}$ , and the range of values is from 90 degrees to −90 degrees. $O_{r}$ is the angle between the horizontal plane of the earth and the positive $X_{s}$ . $O_{r}$ is positive when the positive $Z_{s}$ rotates toward the positive $X_{s}$ , and the range of values is from 180 degrees to −180 degrees.

In Android operating system, four different sampling frequencies (fastest, game, normal, and UI) of sensors can be selected. The values of the four frequencies are not constant and depend on the computational workload of the smartphone. The fastest sampling frequency is selected and can reach 50 Hz normally. The acceleration, gyroscope, and orientation signals were sampled at 25 Hz and stored on a Secure Digital (SD) card in the smartphone. Table 1 showed the descriptions of activities performed by a subject. Ten healthy volunteers (6 males, age: $25 \pm 5$ years, body mass index (BMI): $23.2 \pm 2.7$ kg/m² and 4 females, age: $23 \pm 3$ years, BMI: $21.5 \pm 2.2$ kg/m²) participated in the study.

Table 1

Description of activities performed by a subject.

Activity tasks	Key	Description
Sitting	Si	The subject sits up on an upholstered chair with armrest (seat height: 48 cm).
Lying	L	The subject lies down on a bed on his back (bed height: 50 cm).
Standing	St	The subject remains standing for 5 s.
Lie-to-sit	L-Si	Initially lying on the bed, after 5 s, the subject sits up and remains sitting on the bed for 5 s.
Sit-to-lie	Si-L	Initially sitting up in the chair, after 5 s, the subject lies down and remains lying on the bed for 5 s.
Sit-to-stand	Si-St	Initially sitting up in the chair, after 5 s, the subject stands up and remains standing for 5 s.
Stand-to-sit	St-Si	Initially standing, after 5 s, the subject sits down and remains seated for 5 s.
Walking	W	Initially standing, after 5 s, the subject walks for 10 s at a normal pace.
Walking upstairs	WU	Initially standing, after 5 s, the subject walks upstairs at a normal pace for 12 steps.
Walking downstairs	WD	Initially standing, after 5 s, the subject walks downstairs at a normal pace for 12 steps.
Running	R	Initially standing, after 5 s, the subject runs for 10 s at a moderate speed.
Jumping	J	Initially standing, after 5 s, the subject jumps up along the gravity direction.
Forward fall	FF	Initially standing, after 5 s, the subject simulates fall frontward onto a mattress (thickness = 10 cm).
Right-side fall	FR	Initially standing, after 5 s, the subject simulates fall towards the right side onto the mattress.
Backward fall	FB	Initially standing, after 5 s, the subject simulates fall backward onto the mattress.
Left-side fall	FL	Initially standing, after 5 s, the subject simulates fall towards the left side onto the mattress.

2.2. Feature Extraction

The sliding window approach was employed to divide the sensor signal into smaller time windows (Figure 3). At a sampling frequency of 25 Hz, each window with 50% overlap represents 1.6 seconds. The features were extracted from the sliding windows signals for activity recognition.

(1) Signal Magnitude Vector

${LA}_{3 a}$ is the signal magnitude vector of linear acceleration. ${LA}_{3 a}$ can be represented as

\begin{matrix} L A_{3 a} = \sqrt{L A_{x}^{2} + L A_{y}^{2} + L A_{Z}^{2}} . \end{matrix}

(1)

Likewise,

A_{3 a}

and

G_{3 a}

have used the same computational method.

(2) Tilt angle (TA) of body trunk

TA is defined as the angle between the positive $Y_{s}$ and the gravitational vector, as follows:

\begin{matrix} TA = \frac{L A_{y}}{\sqrt{L A_{x}^{2} + L A_{y}^{2} + L A_{z}^{2}}} . \end{matrix}

(2)

Twenty signals derived from the acceleration, angular velocity, and orientation are listed as follows:

A_{x}

A_{y}

A_{z}

A_{3 a}

G A_{x}

G A_{y}

G A_{z}

G A_{3 a}

L A_{x}

L A_{y}

L A_{z}

L A_{3 a}

G_{x}

G_{y}

G_{z}

G_{3 a}

O_{a}

O_{p}

O_{r}

, and TA.

Figure 3

The sliding window of the data from the accelerometer, the gyroscope, and the orientation sensor during running. The window slides one time such as the blue-dotted rectangle moves to the blue solid one.

As shown in Table 2, seven kinds of statistical features were defined and described. And all statistical features of each signal were calculated at each window. So 140 features were extracted from the acceleration, angular velocity, and orientation measured by the built-in kinematic sensors of the smartphone. ${LA}_{x} Avg$ represents the average value of $L A_{x}$ over the window, and the remaining 139 features are defined similarly (Table 3).

Table 2

Definitions of statistical features.

Statistical features	Key	Description
Average	Avg	The average value over the window.
Median	Med	The median value over the window.
Standard deviation	Std	The standard deviation value over the window.
Skewness	SK	$SK = \frac{n}{(n - 1) (n - 2)} \sum_{}^{} {(\frac{x_{i} - Avg}{Std})}^{3}$ ,
Skewness	SK	the degree of asymmetry of the distribution over the window.
Kurtosis	K	$K = \frac{n (n + 1) \sum {(x_{i} - Avg)}^{4} - 3 {(\sum {(x_{i} - Avg)}^{2})}^{2} (n - 1)}{(n - 1) (n - 2) (n - 3) {Std}^{4}}$ ,
Kurtosis	K	the degree of peakedness of the distribution over the window.
Interquartile range	IR	$IR = Q_{3} - Q_{1}$ , where $Q_{3}$ , $Q_{1}$ is the 75th and 25th percentiles over the window, respectively.
Percentage of decline	PD	The percentage of point decline in the entire window.

Table 3

Definitions of features.

Features	Description
$A_{x}$ Avg	The average value over the window of $A_{x}$ .
${GA}_{y}$ Med	The median value over the window of ${GA}_{y}$ .
${LA}_{z}$ Std	The standard deviation value over the window of ${LA}_{z}$ .
$G_{3 a}$ SK	The skewness value over the window of $G_{3 a}$ .
$O_{a}$ K	The kurtosis value over the window of $O_{a}$ .
$O_{p}$ IR	The interquartile range over the window of $O_{p}$ .
$O_{r}$ PD	The percentage of decline in the entire window of $O_{r}$ .

2.3. Feature Selection and Classification

A combined algorithm of Fisher's discriminant ratio (FDR) criterion and $J_{3}$ criterion was used for feature selection [19]. Firstly, normalize the features to have zero mean and unit variance. Then the features were ranked by the scalar feature selection, which employs FDR criterion and a cross-correlation measurement between pairs of features. FDR is commonly employed to quantify the discriminatory power of individual features between two classes. The FDR is defined as

\begin{matrix} FDR = \frac{{(m_{1} - m_{2})}^{2}}{σ_{1}^{2} + σ_{2}^{2}}, \end{matrix}

(3)

where

m_{1}

m_{2}

are the mean values of features and

σ_{1}

σ_{2}

are the standard deviation of features in the two classes, respectively.

Then the 10 highest-ranked features were selected. And the exhaustive search method with the $J_{3}$ criterion was employed to select the best combination of two features. The $J_{3}$ criterion is done by the following equations:

\begin{matrix} J_{3} = trac {S_{w}^{- 1} S_{b}}, \end{matrix}

(4)

where

S_{w}

is the within-class scatter matrix and

S_{b}

is the between-class scatter matrix. The respective definitions are

\begin{matrix} S_{w} = \sum_{i = 1}^{c} P_{i} S_{i}, \\ S_{b} = \sum_{i = 1}^{c} P_{i} (m_{i} - m_{0}) {(m_{i} - m_{0})}^{T}, \end{matrix}

(5)

where

P_{i}

denotes the a priori probability of ith class Ci and

S_{i}

is the respective covariance matrix of class Ci. Moreover,

m_{0} = \sum_{i = 1}^{c} P_{i} m_{i}

is the global mean vector.

Furthermore, large values of $J_{3}$ indicate that classifiers have small within-class variance and large between-class distance.

A hierarchical classifiers system was shown in Figure 4. According to level of intensity of physical activity, Classifier 1 was used to differentiate between static and dynamic activity. Likewise, Classifier 3 and Classifier 7 utilized the level intensity of activity to recognize high level of intensity of physical activities, such as falls, jumping, and running. In addition, Classifier 2 employed tilt angle (TA) of body trunk derived from accelerometers to identify lying from sitting/standing.

Figure 4

The classifiers flowchart. All abbreviations in this figure were shown in Table 1; for example, “R” represents “Running.”

Moreover, Classifier 5 was employed to classify jumping and falls. Falls are said to have occurred if at least two consecutive peaks in the signal magnitude vector above a defined threshold are recorded [20]. According to the orientation, falls were divided into the categories of right-side fall (FR), left-side fall (FL), backward fall (FB), or forward fall (FF).

Furthermore, there were no obvious features to classify some certain activities, such as walking, walking downstairs and walking upstairs. In a classifier, all possible combinations splitting the activities were exhaustively formed, and for each combination its $J_{3}$ value was calculated. The combination with the largest $J_{3}$ value was chosen to be the scheme of Classifier 6. Likewise, Classifiers 4, 10 used the same method to choose the combination. Fourteen classifiers for two classes were employed to recognize fifteen activities.

3. Results

3.1. The Features Selected

The scatter graphs of part classifiers were shown in Figure 5. The symbols of pluses and circles indicated points into the two classes of classifiers, respectively. In addition, Figure 5 pointed out the combinations of selected features resulted in well separated classes in the 2-dimensional feature space.

Figure 5

The scatter graphs of part classifiers. (a) The scatter graph of C3. (b) The scatter graph of C4. (c) The scatter graph of C8. Pluses and circles indicated points into the two classes of classifiers, respectively.

As shown in Table 4, twenty-eight well selected features for fourteen classifiers included nineteen features derived from accelerometer, six features derived from gyroscope and three derived from the orientation sensor.

Table 4

The selected features of all classifiers.

Classifier	Top 5 optimal features	Selected features
C1	${LA}_{3 a}$ Avg, $A_{z}$ Std, $A_{3 a}$ Std, ${LA}_{3 a}$ Std, ${LA}_{z}$ Std	${LA}_{3 a}$ Avg, $A_{z}$ Std

C2	TAAvg, TAMed, $A_{y}$ Med, ${GA}_{y}$ Avg, $A_{y}$ Avg	TAAvg, $A_{y}$ Med

C3	${LA}_{3 a}$ Std, ${LA}_{3 a}$ IR, $G_{3 a}$ Std, $G_{3 a}$ IR, ${LA}_{x}$ Std	${LA}_{3 a}$ Std, ${LA}_{3 a}$ IR

C4	$O_{p}$ Std, $A_{z}$ Std, TAStd, ${LA}_{3 a}$ Med, ${GA}_{y}$ Med	$O_{p}$ Std, $A_{z}$ Std

C5	${LA}_{y}$ IR, $A_{3 a}$ Std, $A_{3 a}$ IR, $A_{y}$ IR, ${LA}_{3 a}$ Med	${LA}_{y}$ IR, $A_{3 a}$ Std

C6	${LA}_{y}$ Std, $A_{3 a}$ Std, $A_{y}$ Std, $A_{z}$ SK, ${LA}_{3 a}$ Std	$A_{3 a}$ Std, ${LA}_{3 a}$ Std

C7	$A_{3 a}$ Std, $G_{y}$ Std, $A_{y}$ Std, ${LA}_{y}$ Std, ${LA}_{y}$ IR	$A_{3 a}$ Std, $G_{y}$ Std

C8	$O_{r}$ Avg, $A_{x}$ Avg, $G_{x}$ Avg, $G_{z}$ Avg, $O_{3}$ Med	$O_{r}$ Avg, $A_{x}$ Avg,

C9	$G_{3 a}$ Std, $G_{y}$ Std, TAMed, ${GA}_{z}$ Avg, $A_{z}$ Avg	$G_{3 a}$ Std, $G_{y}$ Std

C10	$A_{z}$ Avg, $O_{p}$ Avg, $A_{x}$ Std, $A_{x}$ IR, ${GA}_{z}$ Avg	$A_{z}$ Avg, $O_{p}$ Avg

C11	$G_{y}$ Std, $A_{x}$ Std, ${LA}_{x}$ Avg, ${LA}_{x}$ SK, ${LA}_{x}$ Std	$G_{y}$ Std, $A_{x}$ Std

C12	$G_{y}$ SK, $G_{y}$ K, ${LA}_{3 a}$ Std, ${LA}_{y}$ Std, ${LA}_{z}$ SK	$G_{y}$ SK, $G_{y}$ K

C13	$A_{3 a}$ Std, ${LA}_{y}$ Avg, $G_{y}$ Avg, TAPD, ${LA}_{z}$ Avg	$A_{3 a}$ Std, TAPD

C14	${LA}_{y}$ SK, ${LA}_{3 a}$ K, ${LA}_{3 a}$ SK, ${LA}_{y}$ Std, $A_{y}$ Std	${LA}_{y}$ SK, ${LA}_{3 a}$ K

In these three sensors, accelerometer was dominant to PA recognition. In addition, gyroscope served well in recognizing the change of body posture such as transitions, and the orientation sensor was effective to detect falls. As illustrated in Figure 6, $O_{p}$ can be used to distinguish between forward fall and backward fall. Likewise, $O_{r}$ can be used to distinguish between left-side fall and right-side fall.

Figure 6

Data obtained during four kinds of falls (forward fall, left-side fall, right-side fall, and backward fall). The dotted red rectangles were amplified to observe details. The features from the orientation sensor were effective to detect falls.

3.2. The Application of PA Recognition in the Smartphone

Figure 6 illustrated the application of PA recognition implemented in the smartphone when a subject changed the motion from motionless to walking. The green line indicated the SVM value of the accelerometer in real time. Five common activities (jumping, running, walking, lying, and motionless) can be represented by the blue line.

3.3. The Recognition Results for All Activities

We used hold-out method [21] to estimate the classifiers, which partitions the data into two mutually exclusive subsets: a training set and a test set. A set of experiments was performed with 10 subjects, with each performing a set of 16 different activities as described in Table 1. 600 windows were randomly divided into two parts: a training set (400 samples) and a test set (200 samples).

The confusion matrix in Table 5 was a representative of recognition errors for the test set. From the confusion matrix, we can see that walking may be misclassified as walking upstairs or walking downstairs, because these activities contain similar movements and frequencies at the lower limbs. In addition, falls may be recognized as sit-to-lie, because the posture of the two activities were lying at last. The accuracies of static, transitions, dynamic, and falls were 98.75%, 94.625%, 91.8%, and 97.63%, respectively. A total recognition rate of 95.03% achieved that our proposed PA recognition system based on the smartphone could accomplish high recognition rates for the fifteen activities. The application of PA recognition was displayed in Figure 7. The green line indicated the SVM value of the accelerometer real time, and the blue line presented the PA.

Table 5

Representative confusion matrix.

Classified as	a	b	c	d	e	f	g	h	i	j	k	l	m	n	o
a	199	4	0	0	0	0	0	0	0	0	0	0	0	0	0
b	1	196	0	0	0	0	0	0	0	0	0	0	0	0	0
c	0	0	193	3	0	0	0	0	0	1	0	0	0	0	0
d	0	0	2	192	0	0	0	0	0	0	0	1	1	4	1
e	0	0	0	0	187	13	0	0	0	0	0	0	0	0	0
f	0	0	0	0	10	185	0	0	0	0	0	0	0	0	0
g	0	0	0	0	3	2	185	8	10	3	0	0	0	0	0
h	0	0	0	0	0	0	7	183	6	1	0	0	0	0	0
i	0	0	0	0	0	0	5	7	181	5	0	0	0	0	0
j	0	0	3	0	0	0	3	2	1	182	7	0	0	0	0
k	0	0	2	0	0	0	0	0	2	8	187	2	0	0	0
l	0	0	0	0	0	0	0	0	0	0	1	197	1	2	1
m	0	0	0	1	0	0	0	0	0	0	1	0	195	0	3
n	0	0	0	3	0	0	0	0	0	0	3	0	0	194	0
o	0	0	0	1	0	0	0	0	0	0	1	0	3	0	195

Key: lying = a; sitting/standing = b; lie-to-sit = c; sit-to-lie = d; sit-to-stand = e; stand-to-sit = f; walking = g; walking upstairs = h; walking downstairs = I; running = j; jumping = k; forward fall = l; right-side fall = m; backward fall = n; left-side fall = o.

Figure 7

The application of PA recognition implemented in the smartphone. The green line indicated the SVM value of the accelerometer real time, and the blue line presented the PA.

4. Discussion

A PA recognition system based on a smartphone was developed. The optimal features derived from the built-in kinematic sensors of the smartphone were selected from 140 features for each classifier.

In the previous studies [22, 23], the recognition rates of walking, walking upstairs, and walking downstairs were low, caused by the similarity of these three activities (Figure 8). In this study, the three classes were well separated. Figure 9 presented the scatter graphs of three activities (walking, walking upstairs, and walking downstairs), employing a combination of $G_{y}$ Std, $A_{3 a}$ Std, and $G_{3 a}$ Std. Pluses, circles, and dots indicated points from walking, walking upstairs, and walking downstairs, respectively.

Figure 8

Data obtained during walking, walk downstairs, and walk upstairs.

Figure 9

The scatter graphs of three activities, employing a combination of GyStd, A3aStd, and G3aStd. Pluses, circles, and dots indicated points from walking, walking upstairs, and walking downstairs, respectively.

The PA recognition system based on the smartphone provided several advantages, such as existing communication module and ubiquitous monitor. The previous studies have used Short Messaging Service (SMS)[10] or Multimedia Messaging Service (MMS) [13] to send information of a fall to a guardian. In an earlier study, we warned falls of the elderly to a guardian by MMS including time, map of suspected fall location, and GPS coordinate. When a fall was detected, an automatic MMS would be sent to preset people. According to the convenient communication module of smartphones, more information of PA could be sent in real time. A remote PA recognition system based on a private cloud platform [24] will be developed in our future work.

The main limitation of our approach is that the smartphone must be worn on the chest. People put phones in the pocket of shirts or pants according to their respective habits, which would cause loose attachment between the body and smartphone resulting in the invalidness of the algorithm eventually. Khan et al. [25] used linear discriminant analysis to recognize PA based on a single triaxial accelerometer that allowed subjects to carry the sensor freely in any pocket without firm attachment. In future work, we plan to develop a PA recognition system based on a smartphone according to wearing habits of users.

5. Conclusion

For this study, the optimal features derived from the built-in kinematic sensors of the smartphone were selected to recognize PA. The results of selected features suggested the accelerometer was significant to PA recognition, and gyroscope and orientation sensor were effective to recognize the change of body posture or detect falls, respectively. Despite the limitations of this system which requires firm attachment of the smartphone to a subject, the experimental results demonstrated the feasibility of utilizing the built-in kinematic sensors of the smartphone to recognize PA.

Footnotes

Acknowledgments

This work was supported in part by the National S&T Major Project of China (no. 2011ZX03005-001), Guangdong Province and Chinese Academy of Sciences Comprehensive Strategic Cooperation Project (no. 2011A090100025), and National Natural Science Foundation of Youth Science Foundation (no. 31000447).

References

Ekelund

Brage

Besson

Sharp

Wareham

N. J.

Time spent being sedentary and weight gain in healthy adults: reverse or bidirectional causality?

American Journal of Clinical Nutrition 2008 88 3 612 617

2-s2.0-51749086146

Pinto

Pouliou

Power

Associations between sedentary behaviour and cardiovascular, haemostatic and diabetic markers in mid-adult life (45 years)

Journal of Epidemiology and Community Health 2011 65 A7 A8

Robie

Falls in older people: risk factors and strategies for prevention

The Journal of the American Medical Association 2010 304 1958 1959

Yuwono

Moulton

B. D.

S. W.

Celler

B. G.

Nguyen

H. T.

Unsupervised machine-learning method for improving the performance of ambulatory fall-detection systems

Biomedical Engineering Online 2012 11 9

Caby

Kieffer

de Saint Hubert

Cremer

Macq

Feature extraction and selection for objective gait analysis and fall risk assessment by accelerometry

BioMedical Engineering Online 2011 10, article 1

2-s2.0-78651420068

10.1186/1475-925X-10-1

Sixsmith

Johnson

A smart sensor to detect the falls of the elderly

IEEE Pervasive Computing 2004 3 2 42 47

2-s2.0-3042640871

10.1109/MPRV.2004.1316817

Khan

Z. A.

Sohn

Abnormal human activity recognition system based on R-transform and kernel discriminant technique for elderly home care

IEEE Transactions on Consumer Electronics 2011 57 1843 1850

Lao

Han

de With

P. H. N.

Automatic video-based human motion analyzer for consumer surveillance system

IEEE Transactions on Consumer Electronics 2009 55 2 591 598

2-s2.0-68949175641

10.1109/TCE.2009.5174427

Preece

S. J.

Goulermas

J. Y.

Kenney

L. P. J.

Howard

Meijer

Crompton

Activity identification using body-mounted sensors—a review of classification techniques

Physiological Measurement 2009 30 4 R1 R33

2-s2.0-65249157400

10.1088/0967-3334/30/4/R01

10.

Lee

R. Y. W.

Carlisle

A. J.

Detection of falls using accelerometers and mobile phone technology

Age and Ageing 2011 40 690 696

11.

Nishiguchi

Yamada

Nagai

Mori

Kajiwara

Sonoda

Yoshimura

Yoshitomi

Ito

Okamoto

Reliability and validity of gait analysis by android-based smartphone

Telemedicine and E-Health 2012 18 292 296

12.

Blázquez Gil

Berlanga

Molina

J. M.

InContexto: multisensor architecture to obtain people context from smartphones

International Journal of Distributed Sensor Networks 2012 2012 15

758789

10.1155/2012/758789

13.

Yin

Falling-incident detection and alarm by smartphone with Multimedia Messaging Service (MMS)

E-Health Telecommunication Systems and Networks 2012 1 1 5

14.

Zhang

Markovic

Sapir

Wagenaar

R. C.

Little

T. D.

Continuous functional activity monitoring based on wearable tri-axial accelerometer and gyroscope

Proceedings of the 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth)

2011

370 373

15.

Bahle

Kunze

Lukowicz

On the use of magnetic field disturbances as features for activity recognition with on body sensors

Proceedings of the 5th European Conference on Smart Sensing and Context (EuroSSC '10)

2010

71 81

16.

Chiang

S. Y.

Kan

Y. C.

Lin

H. C.

Activity recognition by fuzzy logic system in wireless sensor network for physical therapy

Intelligent Decision Technologies 2012 191 200

17.

Jeannet

P. Y.

Aminian

Bloetzer

Najafi

Paraschiv-Ionescu

Continuous monitoring and quantification of multiple parameters of daily physical activity in ambulatory Duchenne muscular dystrophy patients

European Journal of Paediatric Neurology 2011 15 1 40 47

2-s2.0-78651349751

10.1016/j.ejpn.2010.07.002

18.

Bonomi

A. G.

Physical activity recognition using a wearable accelerometer

Sensing Emotions 2011 12 41 51 Philips Research Book Series

19.

Theodoridis

Koutroumbas

Pikrakis

Cavouras

Introduction to Pattern Recognition: A Matlab Approach 2010

Academic Press

20.

Karantonis

D. M.

Narayanan

M. R.

Mathie

Lovell

N. H.

Celler

B. G.

Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring

IEEE Transactions on Information Technology in Biomedicine 2006 10 1 156 167

2-s2.0-30744436064

10.1109/TITB.2005.856864

21.

Devroye

L. P.

Wagner

T. J.

Distribution-free performance bounds for potential function rules

IEEE Transactions on Information Theory 1979 25 5 601 604

2-s2.0-0018520640

22.

Maguire

Frisby

Comparison of feature classification algorithm for activity recognition based on accelerometer and heart rate data

Proceedings of the 9th IT & T Conference

2009

Dublin, Ireland

1 11

23.

Ibrahim

R. K.

Ambikairajah

Celler

Lovell

N. H.

Kilmartin

Gait patterns classification using spectral features

Proceedings of the IET Irish Signals and Systems Conference (ISSC '08)

June 2008

Galway, Ireland

98 102

2-s2.0-67649990843

10.1049/cp:20080645

24.

Fan

Toward ubiquitous healthcare services with a novel efficient cloud platform

IEEE Transactions on Biomedical Engineering 2013 60 1 230 234

25.

Khan

A. M.

Lee

Y. K.

Lee

Kim

T. S.

Accelerometer's position independent physical activity recognition system for long-term activity monitoring in the elderly

Medical and Biological Engineering and Computing 2010 48 12 1271 1279

2-s2.0-78049389001

10.1007/s11517-010-0701-3

Classified as	a	b	c	d	e	f	g	h	i	j	k	l	m	n	o
a	199	4	0	0	0	0	0	0	0	0	0	0	0	0	0
b	1	196	0	0	0	0	0	0	0	0	0	0	0	0	0
c	0	0	193	3	0	0	0	0	0	1	0	0	0	0	0
d	0	0	2	192	0	0	0	0	0	0	0	1	1	4	1
e	0	0	0	0	187	13	0	0	0	0	0	0	0	0	0
f	0	0	0	0	10	185	0	0	0	0	0	0	0	0	0
g	0	0	0	0	3	2	185	8	10	3	0	0	0	0	0
h	0	0	0	0	0	0	7	183	6	1	0	0	0	0	0
i	0	0	0	0	0	0	5	7	181	5	0	0	0	0	0
j	0	0	3	0	0	0	3	2	1	182	7	0	0	0	0
k	0	0	2	0	0	0	0	0	2	8	187	2	0	0	0
l	0	0	0	0	0	0	0	0	0	0	1	197	1	2	1
m	0	0	0	1	0	0	0	0	0	0	1	0	195	0	3
n	0	0	0	3	0	0	0	0	0	0	3	0	0	194	0
o	0	0	0	1	0	0	0	0	0	0	1	0	3	0	195

Classified as	a	b	c	d	e	f	g	h	i	j	k	l	m	n	o
a	199	4	0	0	0	0	0	0	0	0	0	0	0	0	0
b	1	196	0	0	0	0	0	0	0	0	0	0	0	0	0
c	0	0	193	3	0	0	0	0	0	1	0	0	0	0	0
d	0	0	2	192	0	0	0	0	0	0	0	1	1	4	1
e	0	0	0	0	187	13	0	0	0	0	0	0	0	0	0
f	0	0	0	0	10	185	0	0	0	0	0	0	0	0	0
g	0	0	0	0	3	2	185	8	10	3	0	0	0	0	0
h	0	0	0	0	0	0	7	183	6	1	0	0	0	0	0
i	0	0	0	0	0	0	5	7	181	5	0	0	0	0	0
j	0	0	3	0	0	0	3	2	1	182	7	0	0	0	0
k	0	0	2	0	0	0	0	0	2	8	187	2	0	0	0
l	0	0	0	0	0	0	0	0	0	0	1	197	1	2	1
m	0	0	0	1	0	0	0	0	0	0	1	0	195	0	3
n	0	0	0	3	0	0	0	0	0	0	3	0	0	194	0
o	0	0	0	1	0	0	0	0	0	0	1	0	3	0	195

Physical Activity Recognition Utilizing the Built-In Kinematic Sensors of a Smartphone

Abstract

1. Introduction

2. Methods

2.1. Device and Data Collection

2.2. Feature Extraction

2.3. Feature Selection and Classification

3. Results

3.1. The Features Selected

3.2. The Application of PA Recognition in the Smartphone

3.3. The Recognition Results for All Activities

4. Discussion

5. Conclusion

Footnotes

Acknowledgments

References

Classified as	a	b	c	d	e	f	g	h	i	j	k	l	m	n	o
a	199	4	0	0	0	0	0	0	0	0	0	0	0	0	0
b	1	196	0	0	0	0	0	0	0	0	0	0	0	0	0
c	0	0	193	3	0	0	0	0	0	1	0	0	0	0	0
d	0	0	2	192	0	0	0	0	0	0	0	1	1	4	1
e	0	0	0	0	187	13	0	0	0	0	0	0	0	0	0
f	0	0	0	0	10	185	0	0	0	0	0	0	0	0	0
g	0	0	0	0	3	2	185	8	10	3	0	0	0	0	0
h	0	0	0	0	0	0	7	183	6	1	0	0	0	0	0
i	0	0	0	0	0	0	5	7	181	5	0	0	0	0	0
j	0	0	3	0	0	0	3	2	1	182	7	0	0	0	0
k	0	0	2	0	0	0	0	0	2	8	187	2	0	0	0
l	0	0	0	0	0	0	0	0	0	0	1	197	1	2	1
m	0	0	0	1	0	0	0	0	0	0	1	0	195	0	3
n	0	0	0	3	0	0	0	0	0	0	3	0	0	194	0
o	0	0	0	1	0	0	0	0	0	0	1	0	3	0	195