Sage Journals: Discover world-class research

Abstract

Acceleration sensor is extensively used in the field of human activity recognition, since it provides better recognition rate of human activity. Based on the principle of molecular attribute, a simple and adaptive activity recognition method is proposed using the acceleration data flow, which constitutes a serial activity, when the acceleration data are treated as the material flow with certain molecular structure. Then five molecular attributes including relative molecular mass, density, internal forces in a molecule, molecule stability, and attraction between molecules are introduced to recognize six human activities, since the closer molecular attribute means the more similar activity. Based on the calculated molecular attributes, a reliability-based voting method for human activity recognition is developed. Since each activity has respective motion cycle, a sliding window with variable sizes is put forward to enhance the recognition rate. Furthermore, adaptive incremental learning is designed to adapt to the different users. The long-time experimental results show that the proposed method is rather accurate and robust for different crowds. The average recognition rate achieves 97.2% for six human activities including walking, jogging, running, going upstairs, going downstairs, and sitting down.

Keywords

Activity recognition acceleration sensor molecular feature variable sliding window incremental learning

Introduction

Recently, wearable devices have been widely used in human activity monitoring, such as the monitoring of falls for the elderly, real-time patient monitoring, and the recording of motion data.^1–6 To detect human activities, all kinds of sensors like accelerometers, gyroscopes or magnetometers, barometer are equipped to wearable devices. Then the devices are placed on human body at different positions so that human activities can be comprehensively judged based on data collected by these sensors.^7–9 For convenience, it is also common that the sensors are also attached to a certain key position on human body.^10–12

Among various sensors, accelerometry has proved itself as a practical, inexpensive, and reliable choice for human activity recognition.^13,14 To enhance the recognition rate, feature extraction and pattern recognition are key techniques using the acceleration data. Feature extraction method can be categorized into statistical and structural features. Statistical features mainly include mean, median, time domain, frequency domain, standard deviation, and so on. Statistical features extract quantitative properties of sensor data, while structural features use the relationship among the mobile sensor data. Some pattern recognition techniques including classifiers,¹⁵ incremental learning, and deep learning are introduced for the activity recognition. The common classifiers include decision tree,¹⁶ support vector machine (SVM),^17–19 Bayesian classifier,^11,20 neural network,²¹ artificial neural network,^22,23 K-nearest neighbor (KNN), hidden Markov model, and so on. On the other hand, some mixed classifiers are also put forward to pursuit the recognition performance. In Gillette and Silverman,²⁴ to recognize the different activities including walking, going upstairs, and going downstairs for both old and young people, Muscillo et al. put forward the combined Kalman filter and Bayesian classifier which achieves favorable recognition performance.

The traditional batch learning methods recognize activities with fixed models, which are unable to adapt to the dynamic changes of human activity. Therefore, the sliding window²⁵ and incremental learning methods have been proposed to address these problems. Banos et al.²⁶ investigated the effect of sliding window size on the recognition rate and determined the optimal window size. Abdallah et al.²⁷ proposed a novel man–machine cooperative incremental learning method. When the classifier fails to identify the fuzzy activity, the program automatically starts incremental learning. Above proposed methods often heavily rely on heuristic hand-crafted feature extraction, which could hinder their generalization performance. However, the deep learning^28–30 reduces the dependency on feature extraction and achieves better performance by automatically learning high-level representations of the sensor data.

When the acceleration data in $x$ , $y$ , and $z$ axes are projected into three-dimensional space, a specific scatter map can be formed. Due to the high similarity of the acceleration data sequence collected from multiple cycles of a specific action, the shape and size of the scatter map have a high similarity. However, there are obvious differences between three-dimensional scatter maps of different actions. The molecular shapes and structures of different substances in the physical world are different. Therefore, in this article, specific actions are analogized to substances, and the three-dimensional spatial scatter plot of acceleration is analogized to the molecular structure of substances. The acceleration data flow constituting a serial activity is considered as a material flow with certain molecular structure. By extracting the statistical and structural features of the molecular attributes, the activity recognition method is designed. The molecular attributes including the relative molecular mass, density, attraction between molecules, internal forces, and molecule stability are defined as features. Then a molecular attribute reliability-based voting (RBV) scheme is designed for human activity recognition. In addition, a sliding window with variable sizes is put forward, since the acceleration data generated by different serial activity types differ greatly in volume. An incremental learning is also derived to adapt to the different users. To reduce the computational complexity of the designed recognition algorithms and enhance the recognition rate, an activity recognition method is proposed based on the principle of molecular attributes. The main contributions of this article are as follows:

The molecular attributes are designed to recognize the human activities using the accelerometer data. The acceleration data flow is considered as a material flow with certain molecular structure. All kinds of features of the molecular attributes are used as classification basis.

By analyzing the credibility of various features and combining the principle of material similarity and compatibility, a voting mechanism-based activity classification method is proposed to improve the recognition performance.

In order to make the proposed activity recognition method adapt to different users, an incremental learning method is put forward. In the method, the samples are adjusted gradually according to the weight of the individual activity data, so that the algorithm can adapt to the different users.

This article mainly presents the human activity recognition method based on the molecular attributes. The rest of this article is structured as follows. Section “Data acquisition” introduces the device for data acquisition. Section “Definition of molecular attributes” describes the definition of the molecular attribute. Section “Voting-based classification” presents the voting-based classification. Then section “Incremental learning” derives the incremental learning method. Section “Experimental analysis” analyzes the experimental results. The conclusion is presented in section “Conclusion.”

Data acquisition

As shown in Figure 1, the data acquisition system consists of four modules which are power supply, accelerometer sensor, bluetooth, and main control module. Among them, the capacity and rated voltage of the supply battery are 250 mAh and 3.3 V, respectively. Bluetooth module realizes communication based on bluetooth version 4.0 and uses low power chip cc2541 which can reach communication range of 20 m. The main control unit is control chip MSP430. To identify basic activities like walking, running, or standing, an accelerometer sensor is attached to a specific limb like the waist. Since accelerometer measures the change of velocity over time in a three-dimensional space, so it is used to recognize the different activities. The main control chip drives the acceleration sensor by the serial peripheral interface (SPI) bus. Then the acquired accelerometer data are sent to a central server where the activity recognition takes place by the bluetooth module. The size of whole hardware device powered by the power supply module is only 3.1 cm × 3.1 cm × 2.2 cm. The size of the device is enough smaller to reduce the interference by external factors.

Figure 1.

Illustration of data acquisition device.

Sampling frequency of the sensors has an impact on the activity recognition results during the data acquisition. The higher sampling frequency produces more data per second, which might be necessary to detect short-term events. However, the higher sampling frequency requires more communication overhead and leads the increasing in communication delay. Typical sampling frequency is varied from 20 to 100 Hz. In our system, the sampling frequency is set at 60 Hz for recognizing six kinds of activities including walking, jogging, running, going upstairs, going downstairs, and sitting down. In the following experiments, 20 subjects in good health are participated. Five hundred cycles of accelerometer data are collected for each activity, while the sensors are placed at the abdomen of each subject in the same orientation.

Definition of molecular attributes

Effective feature extraction is crucial for the subsequent activity classification using the acceleration data. However, invalid feature is useless for the activity recognition and would increase the amount of computation. In our proposed method, the three-dimensional acceleration data are directly used to constitute the molecular attributes. Figure 2 shows the data distribution of similar activities including walking, jogging, and running in three-dimensional space. It can be observed that the data of each activity are grouped within a certain region. The data of similar activities are located in adjacent regions with partial overlapping. Each activity has different accelerometer data distribution density.

Figure 2.

Distribution of acceleration data in three-dimensional space.

In our proposed scheme, the acceleration data flows representing different activities in three-dimensional space are regarded as the material flows with different molecular structures. When each acceleration data point is considered as an atom, the data series representing a full activity cycle constitute a molecule, which is referred to as an activity molecule. Each molecule represents a cycle of a kind of activity such as a step in walking. Then the various molecule attributes such as relative molecular mass, density, attraction between molecules, internal forces in a molecule, and molecular stability are all defined and used to depict the feature of activity recognition. Assuming that $D$ denotes the sample set of a series of human activities and can be written as

D = {A^{s_{1}} (x_{1}), A^{s_{2}} (x_{2}), \dots, A^{s_{n}} (x_{n})}

(1)

where $x_{i} = [x_{i} y_{i} z_{i}]$ is sampled accelerometer data in three-dimensional space, $i = 1, 2, \dots, n$ . $A^{s_{i}} (x_{i})$ represents that the activity of the atom $x_{i}$ is $A^{s_{i}}$ , $s_{i} \in {1, 2, \dots, S}$ . $s_{i}$ denotes the activity order. The detailed definitions of molecular attributes are listed as follows:

Relative molecular mass. In a real physical world, the molecule of a same material has a unique molecular mass. Each activity has its feature similar with the molecular mass, so the relative molecular mass is used to depict the activity feature. The relative molecular mass, denoted as $Ram$ , can be calculated by

Ram = a \bar{x} + b \bar{y} + c \bar{z}

(2)

where $a$ , $b$ , and $c$ are constants and used to adjust the weight in different directions. By setting the weights $a$ , $b$ , and $c$ to different values, the obtained relative molecular mass can be guaranteed to be unique. As long as the values of $a$ , $b$ , and $c$ are different, it can be achieved. $\bar{x}$ , $\bar{y}$ , $\bar{z}$ are the centroid of the molecule in each direction and given by

{\begin{matrix} \bar{x} = \frac{\sum_{i = 1}^{n} x_{i}}{n} \\ \begin{matrix} \bar{y} = \frac{\sum_{i = 1}^{n} y_{i}}{n} \\ \begin{matrix} \bar{z} = \frac{\sum_{i = 1}^{n} z_{i}}{n} \end{matrix} \end{matrix} \end{matrix}

(3)

2. Density. Density means the mass of material in unit volume and varies from one material to another. Since the density is also an important molecule attribute, it is selected as a feature of activity recognition. As one of the molecule attributes, the density is denoted by $ρ$ and can be calculated by

ρ = \frac{M}{V}

(4)

where $M$ and $V$ represent the quality and volume of material, respectively. $M$ and $V$ are further given by

{\begin{matrix} M = \frac{Ram}{N_{a}} \\ \begin{matrix} V = \frac{4}{3} π r^{3} \end{matrix} \end{matrix}

(5)

where $N_{a}$ denotes a constant in physics and $r$ represents the average molecule radius, which is calculated by

r = \frac{\sum_{i = 1}^{n} \sqrt{{(x_{i} - \bar{x})}^{2} + {(y_{i} - \bar{y})}^{2} + {(z_{i} - \bar{z})}^{2}}}{n}

(6)

3. Internal force in a molecule. When two atoms in a molecular are enough far, the main manifestation is attraction between atoms. However, when two atoms are closer, they act as repulsion. Different activities have different spacing of acceleration data in time sequence, so the spacing between atoms is also different. The internal force in a molecule can well characterize the interatomic correlation. The interaction between atoms, known as the van der Waals (VDW) force, is considered as an attribute of the molecular. The average $VDW$ in an activity cycle can be calculated by

VDW = \frac{\sum_{i = 1}^{n - 1} (\frac{A}{d {(i, i + 1)}^{p}} - \frac{B}{d {(i, i + 1)}^{q}}) + \sum_{i = 1}^{n} (\frac{A}{d {(i, o)}^{p}} - \frac{B}{d {(i, o)}^{q}})}{S}

(7)

where $A$ , $B$ , $p$ , and $q$ are constants; $d (i, i + 1)$ denotes the Euclidean distance between two neighboring atoms; $d (i, o)$ represents the Euclidean distance between the ith atom and the centroid of the molecule. $n$ denotes the number of acceleration acquisition points, that is, the number of atoms, in a total of $S$ complete activity cycles. The first and second half of the formula represents the total interaction force between atoms and the total interaction force between atoms and the center of mass, respectively.

4. Molecule stability. Atoms in matter always vibrate all the time. The greater amplitude and frequency means the worse stability of matter. When the amplitude of acceleration data for a certain activity is larger, the material stability corresponding to the activity is worse. The stability can be calculated by

Stability = \frac{\sum_{i = 1}^{n - 1} {(d (i, o) - r)}^{2}}{n}

(8)

where $d (i, o)$ denotes the Euclidean distance between the ith atom and the centroid of the molecule.

5. Attraction between molecules. In physical world, universal attraction exists between any two molecules. The attraction decreases as the distance between two molecules increases. For two similar activities (such as walking and jogging), the closer molecule configurations yield greater attraction. The attraction is obtained with

F = \frac{G M_{rt} M_{s}}{d (rt, s)}

(9)

where $M_{rt}$ is the quality of the recognized accelerometer data, $M_{s}$ is the quality of the sample data, and $G$ is a physical constant. $d (rt, s)$ denotes the Euclidean distance between the centroid of two molecules. The attraction between molecules is an important index for subsequent voting classification. When the attraction between the acceleration data and the sample is maximum, the gravitational vote is cast to the sample. As shown in Figure 3, the attraction F2 between the recognized activity molecule and the standard molecule of jogging is greatest; the jogging gets the attraction vote. Therefore, it is shown that the real probability of acceleration data is caused by jogging.

Figure 3.

Illustration of an attraction-based voting.

Voting-based classification

In this section, a voting-based classification method based on the molecule attribute is proposed. First, vote assignment of feature is designed according to eigenvalue variance. Then the voting classification process is proposed to recognize the different activities using variable sliding window.

Vote assignment

Classifier is very important for activity recognition, since excellent classifier can greatly improve the recognition rate. In our scheme, a reasonable RBV classification method is designed to recognize the human activity. The molecular attributes include relative molecular mass ( $Ram$ ), density ( $ρ$ ), van der Waals ( $VDW$ ) force, molecule stability, and attraction between molecules. Except that the attraction is a structural feature which depicts the relationship between the accelerometers, the other four attributes are statistical features extracting the quantitative properties. The features extracted from the sample data are fluctuated for the individual difference of human activity. When the fluctuation of feature is greater, the reliability of the feature for voting classification is smaller. The fluctuation of feature can be evaluated by the mean square error (MSE) of the sample feature. When the weights are set to $a = 1$ , $b = 2$ , and $c = 3$ for obtaining the relative molecular mass, Figure 4 shows the MSE of the walking activity with 20 different subjects. It is shown that the stability has the biggest MSE among four attributes, so the reliability of stability is lowest. However, the attribute of relative molecular mass has the smallest MSE, so the reliability of relative molecular mass is highest.

Figure 4.

Mean square errors of different attribute features.

It is unreasonable to assign the same votes to each feature because each feature has different reliability and contribution to activity classification. Therefore, the vote allocation method is designed based on the MSE of the attribute feature from 20 subjects in our scheme. Considering the walking activity as an example, the MSEs of the four features with respect to relative molecular mass, density, internal force, and stability are denoted by $v_{1}$ , $v_{2}$ , $v_{3}$ , and $v_{4}$ , respectively. Therefore, the vote assigned to each feature is denoted as $vote s_{i}$ and calculated by

vote s_{i} = \frac{4 \frac{1}{v_{i}}}{5 \sum_{i = 1}^{4} \frac{1}{v_{i}}} \cdot NV

(10)

where $NV$ indicates the total votes, $i = 1, 2, 3, 4$ . However, votes⁵ assigned to the feature attraction is always the same and expressed by

vote s_{5} = \frac{1}{5} NV

(11)

Classification of activities

Using the extracted activity feature of the acceleration data, the eigenvalue of the recognition is obtained and used to vote and classify the activities. Figure 5 illustrates the whole voting process with the real-time acceleration data. The five features are first extracted from the real-time acceleration data and compared with the sample eigenvalue for voting. Then the activities are recognized using the voting results. Figure 5 shows the acceleration waveform calculated by the quadratic sum of the data for walking activity. It can be seen that the acceleration waveform of each activity cycle is regular, since each waveform period represents a complete activity. In the whole cycle of walking activity, 41 acceleration data are produced, but the number of acceleration data for a complete cycle of different activities is not the same. The average data volumes in a cycle (i.e. the average number of atoms in a molecule) of different activities are analyzed and listed in Table 1.

Figure 5.

Illustration of the voting and classification procedure.

Table 1.

Average volume of acceleration data for different activities.

Activity	Walking	Jogging	Running	Going upstairs	Going downstairs	Sitting down
$N_{i}$	41	34	27	45	39	36

In order to effectively use acceleration data to improve the recognition rate, the design of sliding window is particularly crucial. It can also seen from Table 1 that the number of acceleration data in a complete activity period is quite different. When the size of sliding window is invariable, the recognition rate would degrade. Accordingly, a sliding window method with variable size is proposed to improve the recognition performance. The specific procedure of the proposed recognition method is described as follows:

Step 1: Different activity molecules are ranked with the number of atoms ( $N_{i}$ ) in ascending order. The order is listed as follows: running ( $i = 1$ , $N_{1} = 27$ ), jogging ( $i = 2$ , $N_{2} = 34$ ), going downstairs ( $i = 3$ , $N_{3} = 39$ ), walking ( $i = 4$ , $N_{4} = 41$ ), sitting down ( $i = 5$ , $N_{5} = 36$ ), and going upstairs ( $i = 6$ , $N_{6} = 45$ ).

Step 2: Assuming that the activity order is identified as $i$ , the size of the sliding window is set to $N_{i} + 3$ . Initially, $i = 1$ , (it is a running activity), the size of the sliding window is also set slightly greater than $N_{i}$ on purpose. The feature of the acceleration data in the sliding window is extracted and compared with the feature of sample activity for voting.

Step 3: Using the voting-based classification, the activity is recognized with the extracted feature from acceleration data. When the recognition is completed, the vote $p_{i}$ generated by the activity is added to the set PV, and $i = i + 1$ . If $i < 7$ , step 2 is executed. If $i = 7$ , the maximum in the set PV is selected. The corresponding activity is considered as the activity identified by the method. If there are more than one maximum value, the activity is recognized using cosine similarity of feature. Then the sliding window is moved back $0.3 \times N_{i}$ and guarantees a 70% overlap between the sliding window and the original, and proceed to step 4.

Step 4: Set $i = 1$ , proceed to step 2.

Using the walking activity as the example, the features of real-time data are compared with those of standard sample. If the relative error $δ$ stays below the preset threshold re (i.e. $| δ | < re$ ), the corresponding votes are cast to the walking activity. The relative error is defined as

δ = \frac{v_{s} - v}{v_{s}} \times 100 %

(12)

where $v_{s}$ and $v$ indicate the feature values of the standard sample and the real-time data to be recognized, respectively. If the vote is not casted to the walking activity, the sliding window size is changed to try on other activities. The voting follows the best of five, as shown in Table 2 (in which NV denotes the total votes). When the number of votes lies in (0.6NV, 0.8NV), the correct activity has been recognized using the best of five. However, the incremental learning should also be used for enhancing the performance adaptively among different subjects, since the differences of subject do exist when performing the same activity.

Table 2.

Voting results.

Number of votes	[0, 0.2NV)	[0.2NV, 0.4NV)	[0.4NV, 0.6NV)	[0.6NV, 0.8NV)	[0.8NV, NV)
Activity judgment	No	No	No	Yes and incremental learning	Yes

Incremental learning

The goal of incremental learning is to increase the adaptive ability of the proposed method. The main idea of the incremental learning method is to gradually integrate new samples into old samples in proportion and adjust the samples. When the users use the samples for a long time, the samples would tend to be personalized users. Therefore, the incremental learning can effectively solve the problem of low recognition rate caused by the individual differences of users. It can be seen that the incremental learning is required when the voting interval is in [0.6NV, 0.8NV). The incremental learning is mainly realized by adjusting the sample feature in our scheme. When there is a difference between the real-time feature and the sample, it may be that different users perform the same activity. However, due to the activity difference of different users, the feature of the samples is required to be adjusted gradually to adapt to the new users. The features that require incremental learning include relative molecular mass, density, internal force in a molecule, and stability, while the attraction does not require incremental learning. The incremental learning is described as follows:

Relative molecular mass. In incremental learning, the relative molecular mass of the real-time activity is denoted by $rRam$ which is integrated into the standard sample. Therefore, the new relative molecular mass (denoted as $newRam$ ) of the standard sample is given by

newRam = \frac{S \cdot Ram + rRam}{S + 1}

(13)

2. Density. The density always changes with the radius $r$ in incremental learning. Therefore, the updating on $r$ is obtained with

newr = \frac{d (o, o') + r + \tilde{r}}{2}

(14)

where $r$ denotes the radius of the standard sample molecule, $\tilde{r}$ denotes the radius of the molecule to be recognized, and $d (o, o')$ denotes the distance between the centroid of the standard molecule and the molecule to be recognized.

The molecular mass can be recalculated according to the updated relative molecular mass $newRam$ . Assuming the updated mass and volume are denoted as $newM$ and $newV$ , respectively, the updated density $new ρ$ would be given by

new ρ = \frac{newM}{newV}

(15)

3. Internal force and stability. The internal force within the molecule and molecule stability can be obtained with

newv = \frac{S \cdot v_{s} + v}{S + 1}

(16)

where $v_{s}$ and $v$ also indicate the feature values of the standard sample and the real-time feature to be recognized, respectively.

Experimental analysis

In our experiments, the recognition rate of the proposed method is calculated using the acceleration sensors fixed on the abdomen of human. Sample data of 20 subjects were obtained at the early stage of the experiment. By analyzing the reliability of the feature of the sample data, votes assigned to each feature are listed as Table 3. In Table 3, the total votes $NV$ are 20, while the vote assigned to the attraction is always $0.2 NV$ .

Table 3.

Assigned votes using RBV.

Activity	Ram	$ρ$	VDW	Stability	$F$
Walking	6.4	4.9	3.2	1.5	4.0
Jogging	5.6	5.1	3.6	1.7	4.0
Running	3.6	5.5	3.4	3.5	4.0
Going upstairs	5.2	4.5	2.3	4.0	4.0
Going downstairs	4.6	3.9	5.1	2.4	4.0
Sitting down	4.3	6.2	2.3	3.2	4.0

RBV: reliability-based voting; VDW: van der Waals.

When the sample data from 20 subjects were acquired in the first stage, another 15 healthy subjects were selected to validate the proposed method. Each of the subjects installing the acceleration sensors on their abdomen would perform six activities including walking, jogging, running, go upstairs, go downstairs, and sitting down in their own individual natural way. Each subject finished 500 cycles for each activity; then the acceleration data were collected at a frequency of 60 Hz in the experiments.

Effects of voting

The reliability of different features is different, so each feature has different contributions to activity classification. In the recognition method, RBV method is proposed. Other than the RBV method, the votes could also be evenly allocated on each feature (referred to as average voting, AV). In the AV method, each vote assigned to the five features is always fixed at $0.2 NV$ , since the total number of votes is 20 ( $NV = 20$ ). The performances of the recognition rate are listed in Table 4 with two kinds of different voting methods, when the relative error between the feature of sample and that of real-time data to be identified is set to 16% (i.e. $δ ⩽ 16 %$ ). As shown in Table 4, the recognition rate of AV for walking is 95.6%, which is smaller than that of RBV of 99.4%. It is also shown that the recognition rate of each activity is increased, when the voting methods are adjusted from AV to RBV. The proposed RBV gives a 3.8% increase in the average recognition rate compared with the AV method.

Table 4.

Average recognition rates with two different voting methods (RBV and AV).

Activity	AV (%)	RBV (%)
Walking	95.6	99.4
Jogging	90.2	95.3
Running	94.6	98.6
Going upstairs	89.7	94.3
Going downstairs	93.1	96.2
Sitting down	97.5	100
Average recognition	93.5	97.3

RBV: reliability-based voting; AV: average voting.

Variable sliding window

The influence of sliding window on performance is also evaluated. The traditional sliding window is fixed, so its sliding window size is generally determined by the number of acceleration data of a complete activity. It can be seen from Table 1 that the fixed sliding window size is set to 45, while the variable sliding window size always changes according to the steps introduced in section ‘Classification of activities.’ We conducted tests on 15 subjects for over 5 h. When the relative error between the sample and real-time data feature is set to 16%, the experiment results are shown in Figure 6. It is shown that the proposed method can effectively improve the recognition rate using variable sliding window, since the variable sliding window can automatically adjust the size of sliding windows based on the each activity. Therefore, the extracting of activity feature and the following voting classification would be more accurate. However, the feature extracting and voting classification are not enough accurate for the fixed sliding window size, when the number of acceleration data in one activity cycle is different from the sliding window size.

Figure 6.

Recognition performances using fixed sliding widow and variable sliding window: (a) walking, (b) jogging, (c) running, (d) going upstairs, (e) going downstairs, and (f) sitting down.

Incremental learning

On the other hand, the recognition rate is increased with the time accumulation due to the using of incremental learning. As can be seen from Figure 6 that the recognition rate of walking is about 91.5% with the variable sliding window at the beginning. However, the recognition rate of walking is almost increased 100% when the incremental learning improves the recognition rate using the 5-h acceleration data. Since the activity of the subjects at the beginning is different from that of the sample, it leads to the low recognition rate. The incremental learning method continually changes the sample feature, so the method can adapt to new users. But the recognition rate does not always increase to 100% and would be stable in a certain recognition rate. Sometimes, the recognition rate would descend after the peak. As shown in Figure 6(f), the activity misjudgments happen for sitting down recognition, so it leads to the wrong incremental learning.

Effects of relative error

Relative error measures the difference between the standard sample feature and the real-time data. When the relative error is smaller, the sample feature is closer to the real-time data, and vice versa. In feature-matching phase, if the relative error is equal to or smaller than the threshold value re, the feature will be voted. The performance of recognition rate is also investigated with different threshold value re. Then the experimental results are the average of 15 subjects and plotted in Figure 7.

Figure 7.

Comparison of recognition rate under different relative errors.

It can be seen that the performance recognition ratio becomes better, when the threshold value re of relative error is increased from 0% to 16%. When $re = 0$ , the recognition ratio of the proposed method is also zero. However, the sample data collectors and testers are not the same subjects, so their activity feature is different. When re=16%, the recognition ratio is highest. However, when re>36%, the recognition ratio is varied from 10% to 20%. When re is too large, the feature except the attraction between molecules becomes invalid in voting. The activity is only judged based on the attraction, thereby showing poor recognition performances, so there are poor recognition performances.

Recognition rate using different classification algorithms

We also performed recognition experiments using some other classification algorithms commonly used for activity recognition, which are, namely, Bayesian network, decision tree, and naive Bayesian (NB) tree. At the same time, the waveform features (including mean, variance, standard deviation, entropy, peak, valley, the correlation among the acceleration data along three axes, the phase difference of the acceleration data along three axes) are extracted for the training and learning of Bayesian network, decision tree, and NB tree classifiers. In the experiment, each activity has 500 complete cycles of data, and the relative error range is set to $δ ⩽ 16 %$ . The relevant recognition results including the number of right and wrong judgments are shown in Tables 5 –7.

Table 5.

Activity classification results using Bayesian network.

Activity	Walking	Jogging	Running	Going upstairs	Going downstairs	Sitting down	Recognition rate
Walking	463	21	0	9	7	0	92.6%
Jogging	25	465	0	10	2	0	93.0%
Running	0	18	474	0	8	0	94.8%
Going upstairs	15	12	0	461	8	4	92.2%
Going downstairs	10	5	2	0	477	6	95.4%
Sitting down	6	4	0	2	5	483	96.6%

Table 6.

Activity classification results using decision tree.

Activity	Walking	Jogging	Running	Going upstairs	Going downstairs	Sitting down	Recognition rate
Walking	492	4	0	3	1	0	98.4%
Jogging	13	475	1	6	5	0	95.0%
Running	2	13	481	0	3	1	96.2%
Going upstairs	17	9	3	470	1	0	94.0%
Going downstairs	20	8	2	2	468	0	93.6%
Sitting down	0	0	3	0	8	489	97.8%

Table 7.

Activity classification results using NB tree.

Activity	Walking	Jogging	Running	Going upstairs	Going downstairs	Sitting down	Recognition rate
Walking	475	12	2	1	10	0	95.0%
Jogging	19	471	5	0	5	0	94.2%
Running	5	14	479	2	0	0	95.8%
Going upstairs	24	7	1	466	2	0	93.2%
Going downstairs	9	23	3	1	458	6	91.6%
Sitting down	0	0	0	4	10	486	97.2%

NB: naive Bayesian.

Table 5 shows the recognition results using Bayesian network. As can be seen that the recognition rate of sitting down activity is 96.6%, which is highest among all activities. The poorest recognition rate is 92.2% for the going upstairs activity. It can be obtained that the average recognition rate using Bayesian network is 94.1%.

When the decision tree is used to classify the activity, the results are listed in Table 6. The recognition rates for six activities including walking, jogging, running, going upstairs, going downstairs, and sitting down are 98.4%, 95.0%, 96.2%, 94.0%, 93.6%, and 97.8%, respectively. The average recognition rate achieves 95.8%. The recognition rate for walking is highest, while the recognition rate for going downstairs is lowest among six different activities.

As can be seen from Table 7 that the recognition rates of NB tree for six activities are 95.0%, 94.2%, 95.8%, 93.2%, 91.6%, and 97.2%, respectively. The average recognition rate for six activities is 94.5%. NB tree has the highest recognition rate for sitting and the lowest recognition rate for downstairs.

Table 8 shows the recognition results of six activities using traditional eigenvalues in the deep learning model deep neural network (DNN). In the DNN model, there are four layers, eight nodes in the input layer (corresponding to eight eigenvalues), sixteen nodes in the hidden layer, eight nodes in the hidden layer, and six nodes in the output layer. The network structure of DNN model is plotted in Figure 8. As can be seen from Table 8 that the recognition rate of DNN model for walking, go downstairs, and sitting down is higher than 95%, and the recognition rate for sitting down is 100%. The average recognition rate of six activities is 95.6%.

Table 8.

Activity classification results using DNN.

Activity	Walking	Jogging	Running	Going upstairs	Going downstairs	Sitting down	Recognition rate
Walking	484	2	4	1	9	0	96.8%
Jogging	4	465	18	2	11	0	95.2%
Running	1	16	473	2	8	0	94.6%
Going upstairs	11	23	1	459	6	0	91.8%
Going downstairs	2	1	1	4	487	0	97.4%
Sitting down	0	0	0	0	0	500	100%

DNN: deep neural network.

Figure 8.

Network structure of DNN model.

Table 9 shows the recognition result of SVM classifier using traditional eigenvalues to classify six activities. SVM is a robust binary classification algorithm, which supports both linear and non-linear classification. The model structure of SVM is shown in Figure 9. There are eight nodes in the input layer (corresponding to the eight types of features), sixteen nodes in the middle layer, and one node in the output layer. The kernels of SVM classifier are Gauss kernels. As can be seen from Table 9 that the recognition rate of SVM classifier for sitting down and running is higher than 95%. The recognition rate of going upstairs is low, only 89.6%. The average recognition rate of six activities is only 93.7%.

Table 9.

Activity classification results using SVM.

Activity	Walking	Jogging	Running	Going upstairs	Going downstairs	Sitting down	Recognition rate
Walking	469	8	7	11	5	0	93.8%
Jogging	8	463	12	5	12	0	92.6%
Running	3	6	477	1	13	0	95.4%
Going upstairs	17	24	4	448	7	0	91.8%
Going downstairs	13	22	2	6	457	0	97.4%
Sitting down	3	0	0	1	0	496	99.2%

SVM: support vector machine.

Figure 9.

Model structure of SVM classifier.

The recognition rates of our proposed method for six activities are 99.0%, 95.2%, 98.6%, 94.2%, 96.4%, and 99.6%, respectively. Among six activities, the recognition performance of sitting down is best, while the recognition performance of go upstairs is poorest. The recognition rate for similar activities (walking, jogging, running) is high using our proposed method, which can effectively solve the recognition difficulty of similar activities. The average recognition rate of the proposed method achieves 97.2%, which is always higher than the average recognition rates of Bayesian network, decision tree, NB tree, DNN, and SVM classifier.

Run time of the proposed method

In the field of human activity recognition, a large number of algorithms are mainly applied to mobile terminals with low power consumption and weak processing ability. If the run time of the algorithm is too high, it will lead to a long delay of the mobile terminal processing chip and cannot adapt to the scene of real-time recognition. When MSP430 microcontroller unit (MCU) is used as the experimental hardware environment to analyze the average time consumed by various algorithms for identifying a complete action, the experimental results are shown in Figure 10.

Figure 10.

Average consumption time.

The average time in the experiment was obtained by classifying and calculating the average value with 50,000 complete action actions. The results show that the biggest average consumption time is DNN classifier, about 183 ms. The time consumed by SVM classifier is slightly lower than that of DNN classifier, but higher than that of other methods. The average run time of our proposed method is only 86 ms and always less than that of the other methods. The computational overhead of our proposed method is mainly concentrated on the feature extraction stage and does not involve a large number of parameters, so the average run time of our proposed method is the smallest.

Conclusion

Although activity recognition is of great significance to life and health, the difficulty of activity recognition lies in the real-time recognition of dynamic data flow. In this article, a wearable device for activity recognition is developed, which is used to collect acceleration data for activity recognition. A simple and adaptive activity recognition method based on the molecular attribute is proposed. In the proposed method, the acceleration space containing the information of a certain activity is considered as the material flow with a certain molecular structure. The statistical and structural features are extracted using the molecular attributes. Accordingly, a RBV mechanism is designed on basis of molecular feature for human activity recognition. Finally, the recognition performances of the proposed method are experimentally verified with an average recognition rate as high as 97.2%, which is higher than that of the Bayesian network, decision tree, NB tree, DNN, and SVM classifier. Our following work will be focused on the human activity recognition using the convolutional neural network (CNN), recurrent neural network (RNN), and other networks.

Footnotes

Handling Editor: Francesc Pozo

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study is supported by National Natural Science Foundation of China Programs (grant no. 60970082) and Zhejiang Key R&D Plan (grant no. 2017C03047).

ORCID iD

Xiaoping Wu

References

Chen

et al . A hand gesture recognition framework and wearable gesture-based interaction prototype for mobile devices. IEEE T Hum-Mach Syst 2014; 44(2): 293–299.

Pyattaev

Johnsson

Andreev

et al . Communication challenges in high-density deployments of wearable wireless devices. IEEE Wirel Commun 2015; 22(1): 12–18.

Thapliyal

Khalus

Labrado

Stress detection and management: a survey of wearable smart health devices. IEEE Consum Electron Mag 2017; 6(4): 64–69.

Wang

Liu

Shahzad

et al . Device-free human activity recognition using commercial WiFi devices. IEEE J Sel Area Comm 2017; 35(5): 1118–1131.

A joint time synchronization and localization method without known clock parameters. Pervasive Mob Comput 2017; 37: 154–170.

Khalifa

Lan

Hassan

et al . HARKE: human activity recognition from kinetic energy harvesting data in wearable devices. IEEE T Mobile Comput 2018; 17(6): 1353–1368.

Lara

Labrador

MA.

A survey on human activity recognition using wearable sensors. IEEE Commun Surv Tut 2013; 15(3): 1192–1209.

Attal

Mohammed

Dedabrishvili

et al . Physical human activity recognition using wearable sensors. Sensors 2015; 15(12): 31314–31338.

Wang

Feng

et al . Motion parameter capturing of multiple mobile targets in robotic sensor networks. IEEE Access 2018; 6: 24375–24390.

10.

Khan

Lee

et al . Accelerometer’s position independent physical activity recognition system for long-term activity monitoring in the elderly. Med Biol Eng Comput 2010; 48(12): 1271–1279.

11.

Gupta

Dallas

Feature selection and activity recognition system using a single triaxial accelerometer. IEEE T Bio-Med Eng 2014; 61(6): 1780–1786.

12.

Fortino

Ghasemzadeh

Gravina

et al . Advances in multi-sensor fusion for body sensor networks: algorithms, architectures, and applications. Inform Fusion 2019; 45: 150–152.

13.

Lara

Perez

Labrador

et al . Centinela: a human activity recognition system based on acceleration and vital sign data. Pervasive Mob Comput 2012; 8(5): 717–729.

14.

Tao

Jin

Yuan

et al . Ensemble manifold rank preserving for acceleration-based human activity recognition. IEEE T Neur Net Lear 2016; 27(6): 1392–1404.

15.

Nazabal

Garcia-Moreno

Artes-Rodrguez

et al . Human activity recognition by combining a small number of classifiers. IEEE J Biomed Health 2015; 20(5): 1–10.

16.

Ghasemzadeh

Jafari

Physical movement monitoring using body sensor networks: a phonological approach to construct spatial decision trees. IEEE T Ind Inform 2011; 7(1): 66–77.

17.

Rodriguez-Martin

Sama

Perez-Lopez

et al . SVM-based posture identification with a single waist-located triaxial accelerometer. Expert Syst Appl 2013; 40(18): 7203–7211.

18.

Zhang

Piccardi

Structural SVM with partial ranking for activity segmentation and classification. IEEE Signal Process Lett 2015; 22(12): 2344–2348.

19.

Chen

Zhu

Soh

et al . Robust human activity recognition using smartphone sensors via CT-PCA and online SVM. IEEE T Ind Inform 2017; 13(6): 3070–3080.

20.

Liu

Wang

et al . Learning structures of interval-based Bayesian networks in probabilistic generative model for human complex activity recognition. Pattern Recogn 2018; 81: 545–561.

21.

Wang

Jiang

et al . An incremental learning method based on probabilistic neural networks and adjustable fuzzy clustering for human activity recognition by using wearable sensors. IEEE T Inf Technol B 2012; 16(4): 691–699.

22.

Zeng

Nguyen

et al . Convolutional neural networks for human activity recognition using mobile sensors. In: International conference on mobile computing, applications and services, Austin, TX, 6–7 November 2014, pp.197–205. IEEE.

23.

Hassan

Huda

Uddin

et al . Human activity recognition from body sensor data using deep learning. J Med Syst 2018; 42(6): 99.

24.

Gillette

Silverman

HF.

An adaptive Kalman-based Bayes estimation technique to classify locomotor activities in young and elderly adults through accelerometers. Med Eng Phys 2010; 32(8): 949–959.

25.

Noor

MHM

Salcic

Wang

KI-K.

Adaptive sliding window segmentation for physical activity recognition using a single tri-axial accelerometer. Sig Process 2017; 38: 41–59.

26.

Banos

Galvez

Damas

et al . Window size impact in human activity recognition. Sensors 2014; 14(4): 6474–6499.

27.

Abdallah

Gaber

Srinivasan

et al . Adaptive mobile activity recognition system with evolving data streams. Neurocomputing 2015; 150: 304–317.

28.

Ronao

Cho

S-B.

Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst Appl 2016; 59: 235–244.

29.

Wang

Zhang

Gao

et al . Device-free wireless localization and activity recognition: a deep learning approach. IEEE Trans Veh Technol 2017; 66(7): 6258–6267.

30.

Plotz

Guan

Deep learning for human activity recognition in mobile computing. IEEE Comp 2018; 51(5): 50–59.

Human activity recognition method based on molecular attributes

Abstract

Keywords

Introduction

Data acquisition

Definition of molecular attributes

Voting-based classification

Vote assignment

Classification of activities

Incremental learning

Experimental analysis

Effects of voting

Variable sliding window

Incremental learning

Effects of relative error

Recognition rate using different classification algorithms

Run time of the proposed method

Conclusion

Footnotes

Declaration of conflicting interests

Funding

ORCID iD

References