Sage Journals: Discover world-class research

Abstract

Human action recognition is a very popular field in computer vision research, and the research results are widely used in people's lives. This paper explores Kinect-based algorithm of human action recognition and applies it to the quality evaluation of cardiopulmonary resuscitation (CPR) operation. At present, the main means of CRP training is through physical auxiliary equipment, which has a large limitation and can only be carried out under specific conditions. CPR simulation training under general conditions can be effectively carried out by means of computer vision, which is a strategy worth popularizing. Using Kinect's powerful skeleton tracking capabilities to obtain key human skeleton data and then perform fine-grained human action analysis. Our model can obtain the critical compression depth (CCD) and compression frequency (CCF) of CPR. Compared with the-state-of-the-art, our algorithm has better stability and real-time performance. At the same time, our algorithm improves the time efficiency by about 60% while guaranteeing high accuracy. In addition, we guide the human body to perform standard movements by setting joint angle specifications.Moreover, our system has been proven to be valid by professional medical staff.

Keywords

Action recognition kinect CPR skeleton tracking

Introduction

Video-based human action recognition is a very popular direction in the field of computer vision,^1–6 because it has been widely used in people's lives, such as video surveillance, human-computer interaction, video understand -ing, smart medicine and so on.In particular, the launch of the Kinect sensor camera by Microsoft has triggered a surge in the use of human skeleton information to analyze human action. Because of its powerful skeleton tracking function, Kinect enables researchers to analyze human action using human skeleton motion trajectories and has been very successful in many fields.

According to the survey, 5,44,000 people die of cardiac arrest every year in China.⁷ Most of them are at home, work place or school, besides there are often onlookers in these places.⁸ Because the prevalence of cardiopulmonary resuscitation (CPR) is far from enough, these patients can not be rescued in time and miss the best time to save, so their lives pass away. The research shows that the most effective first aid measure for the patients with cardiac arrest is to implement effective CPR for them. Within 3–5 minutes, the effective CPR for the patients can achieve a rescue success rate of 49%–75%.⁹ However, more than 90% of the people have not received professional CPR training,and even about 60% of the people have no awareness of the importance of CPR in china.^7–10 At the same time, about 20% of the people have no access to relevant training courses. Therefore, the popularization of CPR is imminent.

The basic process of CPR is: (1) chest compression; (2) open air way; (3) artificial respiration.^8,9 Among them, the most challeng -ing part is chest compression, which requires rescuers to perform chest compression for at least 20 minutes. At the same time, the implementation of chest compression process has a more strict standard requirements, for the non-standard chest compression algorithm, may cause secondary injury to patients. Therefore, it is very important for the specification of CPR chest compression steps. The traditional CPR training algorithm is to complete the training with the help of dummies with monitoring data function. The biggest disadvantage of this algorithm is that ordinary people can not get expensive and professional training dummies. In addition, there is no effective feedback on the operation posture of trainees, and human computer interaction is very inadequate. Thanks to the widely use of sensor camera in the field of computer vision, people can use its powerful skeleton tracking function to capture human skeleton parameters and effectively analyze human behavior. In this paper, a CPR training guidance algorithm is proposed. The CCF and CCD of CPR training process are analyzed in real time through the skeleton data obtained by Kinect, and the incorrect posture in CPR training process is corrected in real time accord -ing to CPR training specification.

The contribution of this paper can be summarized as follows:

In order to improve the popularity of CPR, this paper proposes a convenient and effective algorithm for training and evaluating CPR operations., which has higher time efficiency, precision and stability.

The algorithm in this paper can accurately and steadily obtain the key parameters of CCF and CCD during CPR operation. At the same time, our algorithm improves the time efficiency by 60% compared with Reskin's. On the other hand, our algorithm can also calculate the joint angle in real time and give timely corrective suggestions for irregular posture.

This paper combines computer vision with CPR training, and presents a complete training and evaluation system for CPR training based on human skeleton data, which has been validated by medical professionals.

Related work

At present, a large number of researchers have been working in the field of video-based human action recognition, and have achieved many surprising results,^1–6 making the current human action recognition with very high recognition accuracy. Video-based human action recognition is mainly developed on the understanding of color video. Convolutional neural network plays an excellent role in understanding picture information. Many researchers have made a lot of improvements and improvements on the basis of traditional convolutional neural network. Karen et al.¹¹ proposed a two-stream network architec- tures, which uses color pictures to extract the appearance characteristics of the human body and optical flow information to extract the motion information of the human body. This network architectures not only has superior effects, but also makes a large number of researchers carry out further research on this basis.^12–15 LSTM network architectures has achieved great success in understanding sequence information.^16–18 Swathikiran et al.⁴ proposed an attention mechanism injected LSTM network for video understanding, and achieved amazing results. Due to the wide use of sensor cameras, human action recognition has entered a new level. Many breakthroughs have been made in the analysis of human motion through human skeleton motion trajectory. This way can reduce the impact of background and noise on human action recognition, and human skeleton can well represent human motion under certain conditions. Song et al.¹⁹ proposed an end- to-end LSTM network based on spatio-temporal attention mechanism, which uses human skeleton information to recognize human movements, and achieved good success. Kinect is used to track human bones and extract the behavioral characteristics of human body when performing CPR, which can well analyze the quality of CPR operation. Wang et al.²⁰ proposed a real-time feedback system of CPR training based on Kinect. The system displays CCF and CCD of CPR training process on the computer screen for trainer to adjust their pressing frequency and depth. The author uses some physical markers to place on the hands of trainer and the chest of training models to obtain CCF and CCD, and provides guidance for trainers. Semeraro et al. Hanners et al.²¹ put forward a way of using telephone video stream to assist cardiopulmonary resuscitation. They have reached the means to improve the success of patients' treatment by establishing a direct video phone for callers and EMS dispatchers, and evaluating the CPR operation of rescuers through remote guidance by professional personnel.²² put forward a system called RELIVE, which can extract CCF and CCD of CPR training through the color image and depth image acquired by Kinect. This system can be regarded as a means to evaluate the quality of CPR training. The system also requires trainers to place markers on their hands, which has limitations. Li et al.²³ determined that untrained bystanders are unlikely to be helpful in CPR in drowning events by assessing the impact of fatigue caused by water rescue on the quality of subsequent CPR and the impact of bystander participation on CPR quality in lifeguard rescue. Therefore, professional CPR training is particularly important. Xu et al.²⁴ used Kinect equipment and fitness exercise together, and used some wearable equipment to obtain real-time relevant data of human body movement state, gave real-time guidance of training actions, real-time correction of improper posture in the process of human body fitness exercise, and achieved good human-computer interaction. Christian et al.²⁵ put forward a system called RESKIN. This system uses the skeleton tracking function of Kinect to obtain the skeleton data of human body in real time. By capturing the motion data of human body in the process of CPR training, it uses heuristic swarm intelligence algorithm to search the target parameters CCF and CCD. After a large number of participants' verification,the verification results are effective.Our algorithm uses the skeleton tracking function of Kinect to capture the skeleton data of human motion in real time, extracts the data by means of sliding window mechanism, and uses cosine curve model to represent the real-time state of CPR training, and uses gradient descent optimization strategy to find the optimal parameters of the model, so as to obtain the CCF and CCD we want. Our algorithm has faster fitting speed and solves the problem that the RESKIN fitting model is slow, resulting in the Kinect losing part of the data and obtaining higher accuracy. On the other hand, we use the acquired bone data and spatial geometry knowledge to correct the irregular posture of the trainer in real time. Combined with CCF and CCD, we guide the trainer to carry out the standard CPR training, correct the trainer's wrong posture, wrong press speed and depth put forward a complete CPR training system.

Proposed CPR training model

According to the standard CPR rules, during the CPR training, the trainer’s body will naturally do the simple harmonic motion, and the cosine curve model can well represent the real-time state of the CPR training process. The algorithm in this paper utilizes the powerful human skeleton tracking function of Kinect to capture human skeleton data in real-time,extract human motion feature information, combine wi-th the sliding window mechanism to extract data, fit the cosine curve model, and extract the corresponding model parameters, so as to achieve the purpose of obtaining the desired CCF and CCD, as shown in Figure 1. By combining the acquired human skeleton data with the spatial geometry knowledge and calculating the specific angle information, the purpose of correcting the incorrect CPR training posture of the trainer was realized.

Figure 1.

Display of model fitting effect.

Kinect depth camera can acquire the depth mapping of the scene, and then calculate the three-dimensional spatial coordinates of each pixel in the scene through spatial geometric transformation. With this technique, Kinect can estimate the coordinates of 20 joint points when the human body is located within its lens perspective. As shown in Figure 2, joint position information estimated by Kinect. The origin of each three-dimensional spatial coordinate is the location of the Kinect lens. In addition, Kinect acquires 30 frames of bone data per second, which can track the dynamic information of the human body in real time. As shown in Figure 3, as long as the human body is located in the perspective of Kinect, the human bone data can be acquired in real time.

Figure 2.

Kinect skeleton tracking for human skeleton data.

Figure 3.

Kinect Skeleton tracking diagram.

Motion trajectory feature

Kinect collects the human skeleton data at 30 frames per second. In this paper, the human skeleton data of each frame is collected, and the distance between the human skeleton data and the ground is calculated. Combined with the time change, the motion feature of the trainer in the CPR training process are described. Through experimental comparison, when the human body is facing Kinect, the limbs of the human body are most easily tracked, and the effect is stable and effective. In this paper, three-dimensional spatial data points of shoulder skeleton are extracted as feature data, and the left shoulder is recorded as q, and the right shoulder is recorded as p. in order to simplify the data feature description and reduce the error, the middle points of q and p are taken as key feature data points, and recorded as k=(x,y,z). Select the ground plane as the reference plane, use the depth map of the environment obtained by Kinect, we can get the point cloud data of the current environment, with the help of RANSAC algorithm, the parameters of the ground plane model are easily obtained in the current environment, and recorded as N=(n_x,n_y,n_z,a). According to the geometric relationship between 3 D space point and plane, the distance between 3 D space point and plane can be obtained by equation (1), which is recorded as d.

d = | \frac{n_{x} \cdot x + n_{y} \cdot y + n_{z} \cdot z + a}{\sqrt{{(n_{x})}^{2} + {(n_{y})}^{2} + {(n_{z})}^{2}}} |

(1)

we uses the sliding window mechanism to extract data with a window length of s each time, which is used to fit our target model. Inspired by RESKIN, we also use the way of fetching data forward. Taking a certain time t as an example, the data distribution can be expressed as D={d[t-s],d[t-s + 1],…,d[t-1],d[t]}, and |D|=s. Optimization of cosine curve model parameters driven by dataset of D.

y (t) = A \cos (2 π f · t + ρ) + h

(2)

Where, A represents amplitude, f represents frequency, ρ represents initial phase, h represents offset distance. We extract the cosine curve model parameters fitted by datasets of D to obtain the CCF and CCD that we need. Here, the parameters of each fitting model can approximately represent the CCF and CCD at the current time.

For CPR training, our goal is to achieve real-time guidance for trainers, so there are high real-time requirements. Gradient descent algorithm is used to optimize the parameters of cosine curve model. It needs less training iterations and faster speed to fit our target model, which can meet the real-time requirements with high accuracy and stability.

Body joint angle feature

In the aspect of CPR training posture guidance, we uses the skeleton tracking function of Kinect to obtain the three-dimensional skeleton data points of human body in real-time, and calculates the angle features with the knowledge of space geometry, and corrects the incorrect posture of the trainer in the training process in real time with CPR training posture rules. In 3 D space, clips between adjacent joint points can be represented by vectors. For example, you can represent clips between the shoulder and elbow joints as v.The angle of a joint point θ<v₁,v₂> can be calculated by equation (3).

θ < v_{1}, v_{2} > = arc \cos ((v_{1} \cdot v_{2}) \cdot {(| v_{1} | \cdot | v_{2} |)}^{- 1})

(3)

Where, v₁ represents the segment between p_i and p_j, v₂ represents the segment between p_i and p_k, meanwhile, i ≠ k≠j. v₁·v₂ represents the dot product of two vectors, |v₁| and |v₂| represent the inner product of v₁ and v₂ respectively.

The angle β between the three-dimensional vector v and the plane can be calculated by equation (4).

\begin{array}{l} \sin < v, n > = \cos ((v \cdot n) \cdot (| v | \cdot | n |)^{- 1}) \\ β = arc \sin < v, n > \end{array}

(4)

Where, v is any space vector, n is the normal vector of the ground plane model, v·n is the dot product of the two vectors, |v| and |n| are the inner products of v and n respectively.

Loss function

In this paper, mean square error (MSE) is used to measure the difference between the estimated model and the real sample, and the most appropriate cosine curve model parameters are obtained by minimizing MSE. which is defined as equation (5):

loss = \frac{1}{| D |} \sum_{t = 0}^{| D |} {(d [t] - y (t))}^{2}

(5)

Where, |D| represents the length of sliding window, d[t] represents the absolute distance between k = (x,y,z) and the ground at t time, and y(t) represents the value of cosine curve model at t time.

Parameter optimization

Gradient descent algorithm is a search based optimization algorithm, which is suitable for solving convex optimization problems. Its purpose is to find an appropriate iterative direction for the input data, so that the output value can reach the local minimum value. The gradient update formula is defined as follows:

ω^{1} = ω^{0} - α Δ J (ω)

(6)

Where, ω⁰ represents the current position of parameter ω, ω¹ represents the position of the next step, α represents the learning rate, an ΔJ(ˑ) represents the gradient direction.

At the same time, momentum can be used to reduce the oscillation of the gradient descent process. In the basic gradient descent algorithm, ω¹=ω⁰ – v, of which v = αΔJ(ω). When momentum is added, v is the vector sum of the current gradient decrease and part of the last update v’, so v = αΔJ(ω) + v’m, where m is momentum. If the gradient direction of the last time is the same as that of this time, momentum can accelerate this update, otherwise momentum can decelerate this update. Therefore, adding momentum mechanism to the traditional gradient updating algorithm can make the algorithm better and faster convergence.

The overall algorithm of proposed in this paper is shown in Table 1. Kinect acquires human skeleton data and human motion information in real time at the frequency of 30 frames per second. In the process of CPR training, the frequency and depth of compression will not be consistent throughout the training process. Therefore, through the above mentioned algorithm, the cosine curve model is repeatedly fitted at a time interval of f_update, and the model parameters are extracted to obtain CCF and CCD, which are approximately real-time CCF and CCD, as an index to guide CPR training. On the other hand, through real-time acquisition of three dimen- sional human skeleton data, real-time calculation of the corresponding angle information, extraction of human pose features, combined with CPR training standards and norms, to achieve real-time correction of the trainer's incorrect posture. Through the combination of CCF and CCD numerical guidance and posture specification guidance, we have realized a complete guidance system.

Table 1.

The algorithm flow chart of this paper.

The overall algorithm of proposed
1: Setup Hyperparameters:
2: Time interval →f_update
3: Sliding window size →s
4: Number of iteration optimizations → IN
5: CPR pose specification threshold →ε
6: Kinect gets shoulder joint coordinates q,p at 30 frames per second
7: Loop
8: calculate distance joint q,p with ground→d_q,d_p
9: (d_q+d_p)/2→d_t
10: d_t push into set D
11: for f_update do:
12: extract data→D[s]
13: while Iteration<IN do:
14: feed D[s] into CPR training system
15: fit cosine curve model
16: end while
17: extract cosine curve model parameter A and f
18: transform A and f to CCD and CCF
19: end for
20: calculate joint angle features θ,β via equation (4)
21: if θ < ε and β < ε do:
22: the CPR posture conforms to the standard
23: else
24: correct incorrect CPR posture
25: end Loop

Experiments

Experiments setting

In this section, we will describe the experimental setup of the proposed algorithm in detail, and show the experimental results and analyze the results.We used Matlab 2017 and Kinect V2 to complete all experiments on a computer equipped with Intel i7 8700 CPU. We place the Kinect horizontally at a height of 1.5 m from the ground. The trainer and the simulation patient conduct the training operation at 1.5 m in front of the Kinect lens. The trainer faces the Kinect and presents a kneeling posture, simulat- ing the patient lying horizontally at the trainer's knee. The average CCF was 100 cpm, and the average CCD was 5.5 cm. The sliding window s = 5 second is selected, which corresponds to the frequency of data acquisition per second of 30 frames of Kinect. There will be 150 data points participating in the cosine curve model fitting in each time. We use f_update= 1 second as the time interval to repeatedly fit the target cosine curve model. Every time we update the model parameters, we can approximate them to the real-time CCF and CCD of the time segment between this time and the previous time.

Learning rate and momentum

We need to optimize four parameters, which are A, f, ρ and h. The evaluation criteria were referred to the AHA (American Heart Association) guidelines for CPR. So the CCF reference value was 100–120 cycles per minute, and the CCD reference value was 0.05 m. We set the initial range of parameters as (−0.1,0.1), (0,3), (0,3), (0,1) respectively, and the experimental setting value are 0.05, 1.5, 1.5, 0.5 respectively. For the values of learning rate α and momentum m, we determine their values through experimental comparison, and take the cosine curve model fitting time as the evaluation standard. Set the basic gradient adjustment experiment to determine the relationship between learning rate α and time, as shown in Figure 4. When learning rate α is greater than 1, the model fitting time tends to be stable. Then, we initially set the learning rate α = 1.9, and add momentum m in the gradient adjustment process. The result is shown in Figure 5. As the value of momentum m changes from 0 to 1,the time required for model fitting also decreases steadily. However, when the value of m is greater than 1, the model can not be successfully fitted. Further, we choose momentum m = 0.9 to increase the learning rate. The experimental results are shown in Figure 6 When α = 2.7, the model fitting speed is the fastest. At the same time, considering that the learning rate α value is too large, although the fitting speed is faster, it is also prone to the phenomenon that the model finally oscillates near the optimal solution and cannot be fitted successfully. Therefore, we set the initial learning rate α as 2.7 and momentum m as 0.9, and set its value to half of its current value every 300 iterations.

Figure 4.

Comparison of learning rate.

Figure 5.

Comparison of momentum.

Figure 6.

Determine learning rate.

Time efficiency comparison

In the aspect of comparing the time efficiency of single fitting cosine curve model, we use the same time window and the same set of data to compare the difference in time efficiency of different algorithms under the condition that RESKIN, PSO,²⁴ GWO²⁵ and our proposed algorithms reach the same fitness value. The experimental results are shown in Figure 7. It can be seen from the figure that PSO, GWO and our proposed algorithms are after about 0.1 second, we can reach the preset adaptation value, and our algorithm starts to be close to the preset adaptation value at about 0.01 second, and then approaches the preset adaptation value with a smaller gradient. In contrast, RESKIN only reaches the preset adaptation value at about 0.25 second after the experiment, so we can see that our algorithm is more than twice as efficient as RESKIN in terms of time efficiency. Our goal is to achieve real-time posture tracking and guide CPR training, so real-time is very important.^26,27

Figure 7.

Comparison of time efficiency.

Stability comparison

Our system needs to update the target model in real-time to show the current pressing frequency and pressing depth of trainer. We update our cosine curve model with fupdate as the time interval, and extract the CCF and CCD we want from it. Frequent fitting models have higher requirements for the stability of the algorithm. In this section, we use RESKIN, GWO, PSO and our algorithm to fit the cosine curve model according to the time interval of f_update on the same set of one minute data. This data is a set of uniform data, with an average pressing frequency of 100 cpm and a pressing depth of 5.5 cm. The experimental results are shown in Figures 8 and 9. In one minute, the four algorithms all fit the model 60 times. The results of PSO and GWO are jittery and easy to fall into the local optimal solution. For this system, it is not applicable. RESKIN and our algorithm are relatively stable in terms of pressing frequency or pressing depth. The CCF output of RESKIN and our algorithm is stable at 100 cpm, while the CCD also almost floats at 5.5 cm up and down, rarely with large fitting jittery. The CCF is consistent with CCD, which can find the most suitable model parameters. For this system, it is more stable and applicable. Considering the model fitting time and algorithm stability, our algorithm is superior.

Figure 8.

CCF Stability comparison of algorithms.

Figure 9.

CCD Stability comparison of algorithms.

Model prediction error analysis

In this section, 10 groups of standard data are collected, each group of time is one minute, the average compression frequency is 100 cpm, and the average compression depth is 5.5 cm. At the same time, the compression frequency and depth per second of each group of data are not completely equal, showing the uneven distribution state of speed and depth. We can get the CCF and CCD of each set of data at each time in a minute by model calculation. In addition, we get the error between the predicted value and the ground-truth, averaging to every second, so that we can get the average prediction error, as shown in Table 2. Through comparison, it can be seen that the obvious errors of PSO algorithm and GWO algorithm in compression frequency and compression depth are much higher than those of the other two algorithms, which shows that there is a large error in the actual estimation accuracy of PSO and GWO algorithms. Our algorithm no matter on CCF or CCD, the accuracy of this algorithm is higher than RESKIN.

Table 2.

Average error between CCF(cpm)/CCD(cm) and ground-truth of four algorithms.

Group	algorithm
Group	PSO	GWO	RESKIN	Our
1	7.33/4.51	4.33/3.01	3.27/1.75	2.81/1.53
2	7.79/4.37	4.59/3.18	3.14/2.01	2.79/1.60
3	8.21/4.66	4.51/3.05	2.97/1.96	2.92/1.57
4	7.89/4.77	4.50/2.98	2.88/1.79	2.76/1.61
5	7.66/5.03	4.49/2.97	3.00/1.77	2.57/1.67
6	7.31/4.67	4.72/3.13	3.12/1.96	2.78/1.56
7	7.59/4.59	4.67/3.07	3.32/2.03	2.86/1.52
8	7.88/4.35	4.59/3.11	3.22/1.93	2.94/1.62
9	8.01/5.09	4.55/3.09	3.08/1.85	2.96/1.57
10	8.19/5.21	4.57/3.17	2.95/1.88	2.77/1.49

Posture analysis of CPR training

The CPR training system proposed in this paper, another important branch of its function is the real-time posture guidance. Using the skeleton tracking function of Kinect,we can obtain the three-dimensional space coordinates of shoulder, elbow and wrist in real time, as shown in Figure 2, including 5,67,910 and 11 points. we can obtain the feature points k,r,mark the midpoint of 5, 9 as k, 7 and 11 as r, and use RANSAC algorithm to calculate the parameters of the ground plane model in the current environment, and its normal vector is n. Accord- ing to the CPR training standard, the hands and arms of the trainer should be vertical to the chest plane of the simulated patient at all times during the training, so the β calculated by equation (4) for v_kr and n should be 90°. At the same time, during CPR training, the trainer's arms should be kept straight at all times, so the angles θ_r_,θ_l of the joints where points 6 and 10 are located should be kept at 0°,θ_r,θ_l can be obtained by equation (3). Considering the inherent error of Kinect, we set an error threshold ε. By comparing the absolute error between θ_r,θ_l,β and the standard value, we can evaluate whether the current training posture conforms to the standard and correct the irregular posture in time.

Figure 10(a) is the standard posture, and the absolute errors of θ_r,θ_l,β with standard values meet the preset threshold value of ε = 8°. Figure 10(b) and (c) respectively show that the absolute errors of the current right and left elbow joint angles θ_r,θ_l with standard values exceed the threshold value of ε, Figure 10(d) shows that the absolute errors of both hands elbow joint angles θ_r,θ_l with standard values exceed the threshold value of ε, and Figure 10(e) shows the absolute errors of β with standard values exceeds the preset threshold of ε.

Figure 10.

Analysis of standard posture.

Conclusion

The main purpose of this article is to solve the problem that the popularity of CPR in China is not sufficient and the general public cannot easily obtain CPR training. So we provide a convenient way for CPR training to meet the requirements of the premise, and at the same time, it can be widely obtained and used by the public.

Our algorithm can fit the model faster and meet the real-time requirements. our algorithm has higher CCF and CCD estimation accuracy and meets the estimation accuracy requirements. our algorithm can fit each model more stably and effectively, and can meet the stability requirements of the algorithm well. Finally, compared with RESKIN, our algorithm provides a real-time posture correction enhances the function of human-computer interaction, has a good feedback function in vision, and improves the practical value. Through the operation and evaluation of professional medical staff, it shows that our algorithm is feasible.

Footnotes

Acknowledgements

Thanks for the CPR technical guidance of Ms.Lin Jie in Fuzhou the Second Hospital Affiliated to Xiamen University. The content of this paper is the research results of Innovation team project of FuZhou health committee, subject no. 2018-S-WT3.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: the support of the National Natural Science Foundation of China under Grant 61473330 for this research.

ORCID iDs

Huosheng Xie

Hongwen Luo

Jie Lin

References

Yan

Wang

, et al. PA3D: pose-action 3D machine for video recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2019, pp. 7914–7923.

Varol

Laptev

Schmid

Long-term temporal convolutions for action recognition. IEEE Trans Pattern Anal Machine Intell 2017; 40: 1510–1517.

Tran D, Wang H, Torresani L, et al. A closer look at spatiotemporal convolutions for action recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, IEEE, 2018, pp. 6450–6459.

Sudhakaran

Escalera

, Lanz

LSTA: long short-term attention for egocentric action recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, 2019, pp. 9946–9955.

Piergiovanni

, Ryoo

Representation flow for action recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, 2019, pp. 9937–9945.

Moltisanti

Fidler

, Damen

Action recognition from single timestamp supervision in untrimmed videos. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2019, pp. 9907–9916.

Huang

, et al. A survey on knowledge and attitude of cardiopulmonary resuscitation in a community. Practical Journal of Clinical Medicine 2015; 11: 357–362.

Gao

Zhang

, et al. Research progress of cardiopulmonary resuscitation technology and application at home and abroad. China Modern Medicine 2013; 23: 110–115.

Sun

, et al. Research progress on cardiopulmonary resuscitation training algorithm of medical and nursing personels. Chinese Nursing Research 2014; 35: 4358–4360.

10.

Ying

Lei

, et al. Effectiveness of strengthened training in teaching of emergency cardiopulmonary resuscitation in department of emergency to medical students with different educational history[J]. Chinese Journal of Evidence-Based Cardiovascular Medicine 2015; 17: 211–215.

11.

Simonyan

Zisserman

Two-stream convolutional networks for action recognition in videos. In: Processings of the 28th Annual Conference on Neural Information Processing Systems, NIPS, 2014, pp. 568–576.

12.

Feichtenhofer

Pinz

Zisserman

Convolutional two-stream network fusion for video action recognition. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016, pp. 1933–1941.

13.

Zhao

Liu

Han

, et al. Pooling the convolutional layers in deep ConvNets for video action recognition. IEEE Trans Circuits Syst Video Technol 2017; 28: 1839–1849.

14.

Cho

Foroosh

Spatio-temporal fusion networks for action recognition[J]. In: Proceedings of the 14th Computer Vision Asian Conference on Computer Vision, 2018, pp. 347–364.

15.

You

Jiang

Action4D: real-time action recognition in the crowd and Clutter. In: Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2011, pp. 1849–11858.

16.

Wang H, Feng J. Delving into 3D Action Anticipation from Streaming Videos. arXiv 2019 arXiv-1906, https://arxiv.org/pdf/1906.06521v1.pdf

17.

Chen

Exploring the attention mechanism in LSTM-based Hong Kong stock price movement prediction. Quant Finance 2019; 19: 1507–1509.

18.

Yan

, et al. An attention mechanism based convolutional LSTM network for video action recognition. Multimedia Tools Appl 2019; 78: 20533–20556.

19.

Song

Lan

Xing

, et al. An end-to-end spatio-temporal attention model for human action recognition from skeleton data. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI Press, 2017, pp. 4263–4270.

20.

Wang

Tsai

Chen

, et al. Kinect-based real-time audiovisual feedback device improves cardiopulmonary resuscitation quality of lower-body-weight rescuers. Am J Emerg Med 2017; 36: 577–582.

21.

Ecker

Wingen

Hamacher

, et al. Wetsch evaluation of CPR quality via smartphone with a video livestream – a study in a metropolitan area. Prehospital Emergency Care 2020; 21: 355–362.

22.

Semeraro

Frisoli

Loconsole

, et al. Motion detection technology as a tool for cardiopulmonary resuscitation (CPR) quality training: a randomised crossover mannequin pilot study. Resuscitation 2013; 84: 501–507.

23.

Kan

Guo

, et al. Assessing the quality of CPR performed by a single lifeguard, two lifeguards and a lifeguard with a bystander after water rescue: a quasi-experimental trial. Emergency Medicine Journal 2020; 37: 306–313.

24.

Liu

, et al. Design of fitness coach system based on Kinect 3D somatosensory camera. Modern Electronics Technique 2019; 11: 280–285.

25.

Lins

Eckhoff

Klausen

, et al. Cardiopulmonary resuscitation quality parameters from motion capture data using differential evolution fitting of sinusoids. Appl Soft Comput 2019; 79: 300–309.

26.

, Montes D, Marco A, Stützle T, Birattari M, et al. Frankenstein's PSO: a composite particle swarm optimization algorithm. IEEE Trans Evol Computat 2009; 13: 1120–1132.

27.

Wen L, Zhao D, Xu S, et al..

Improved grey wolf optimization algorithm for constrained optimization problem. J Beijing Union Univ 2010; 26: 212–214.