Low-back musculoskeletal disorders (MSDs) are major cause of work-related injury among workers in manual material handling (MMH). Epidemiology studies show that excessive repetition is one of major risk factors of low-back MSDs. Thus, it is essential to monitor the frequency of lifting tasks for an ergonomics intervention. In the current field practice, safety practitioners need to manually observe workers to identify their lifting frequency, which is time consuming and labor intensive. In this study, we propose a method that can recognize lifting actions from videos using computer vision and deep neural networks. An open-source package OpenPose was first adopted to detect bony landmarks of human body in real time. Interpolation and scaling techniques were then applied to prevent missing points and offset different recording environments. Spatial and temporal kinematic features of human motion were then derived. These features were fed into long short-term memory networks for lifting action recognition. The results show that the F1-score of the lifting action recognition is 0.88. The proposed method has potential to monitor lifting frequency in an automated way and thus could lead to a more practical ergonomics intervention.
Get full access to this article
View all access options for this article.
References
1.
AbawajyJ.ChooK.-K. R.IslamR.XuZ.AtiquzzamanM. (Eds.). (2019). International Conference on Applications and Techniques in Cyber Security and Intelligence ATCI 2018: Applications and Techniques in Cyber Security and Intelligence (Vol. 842). Springer International Publishing. https://doi.org/10.1007/978-3-319-98776-7
AtrsaeiA.SalariehH.AlastyA.AbedinyM. (2018). Human arm motion tracking by inertial/magnetic sensors using unscented Kalman filter and relative motion constraint. Journal of Intelligent & Robotic Systems, 90(1), 161-170.
4.
BernardB. P.Putz-AndersonV. (1997). Musculoskeletal disorders and workplace factors; a critical review of epidemiologic evidence for work-related musculoskeletal disorders of the neck, upper extremity, and low back.
BormannR.HamppJ.HageleM.VinczeM. (2015). Fast and accurate normal estimation by efficient 3d edge detection. 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 3930–3937. https://doi.org/10.1109/IROS.2015.7353930
7.
CallaghanJ. P.McGillS. M. (2001). Intervertebral disc herniation: studies on a porcine model exposed to highly repetitive flexion/extension motion with compressive force. Clinical biomechanics, 16(1), 28-37.
8.
CaoZ.SimonT.WeiS. E.SheikhY. (2017). Realtime multi-person 2d pose estimation using part affinity fields. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7291-7299).
9.
CarreiraJ.ZissermanA. (2017). Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 4724–4733. https://doi.org/10.1109/CVPR.2017.502
10.
ChaK. J.YooK. H.LeeC. U.MunB. M.BaeS. J. (2018). Classification of acoustic noise signals using wavelet spectrum based support vector machine. Journal of Mechanical Science and Technology, 32(6), 2453-2462.
11.
ChenL.WeiH.FerrymanJ. (2013). A survey of human motion analysis using depth imagery. Pattern Recognition Letters, 34(15), 1995–2006. https://doi.org/10.1016/j.patrec.2013.02.006
12.
DesmoulinG. T.PradhanV.MilnerT. E. (2020). Mechanical Aspects of Intervertebral Disc Injury and Implications on Biomechanics. Spine, 45(8), E457–E464. https://doi.org/10.1097/BRS.0000000000003291
13.
DonisiL.CesarelliG.CocciaA.PanigazziM.CapodaglioE. M.D’AddioG. (2021). Work-Related Risk Assessment According to the Revised NIOSH Lifting Equation: A Preliminary Study Using a Wearable Inertial Sensor and Machine Learning. Sensors, 21(8), 2593. https://doi.org/10.3390/s21082593
14.
ElhayekA.de AguiarE.JainA.ThompsonJ.PishchulinL.AndrilukaM.BreglerC.SchieleB.TheobaltC. (2017). MARCOnI—ConvNet-Based MARker-Less Motion Capture in Outdoor and Indoor Scenes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(3), 501–514. https://doi.org/10.1109/TPAMI.2016.2557779
15.
InsafutdinovE.PishchulinL.AndresB., et al.: Deepercut: a deeper, stronger, and faster multi-person pose estimation model. In: European Conference on Computer Vision. Springer, Cham pp. 34–50 (2016)
16.
KayaK.ÖǧüdücüŞ. G. (2018, September). A binary classification model for pm 10 levels. In 2018 3rd International Conference on Computer Science and Engineering (UBMK) (pp. 361-366). IEEE.
17.
KwangYong LimYoung Koon GohF.WeiDongKimDoang NguyenI-MingChenSongHuat YeoBeen Lirn DuhH.ChungGon Kim. (2008). A wearable, self-calibrating, wireless sensor network for body motion processing. 2008 IEEE International Conference on Robotics and Automation, 1017– 1022. https://doi.org/10.1109/ROBOT.2008.4543338
18.
MaherC.UnderwoodM.BuchbinderR. (2017). Non-specific low back pain. 389, 12.
19.
MaimoneA.FuchsH. (2012, March). Reducing interference between multiple structured light depth sensors using motion. In 2012 IEEE Virtual Reality Workshops (VRW) (pp. 51-54). IEEE.
20.
MarrasW. S.ParakkatJ.ChanyA. M.YangG.BurrD.LavenderS. A. (2006). Spine loading as a function of lift frequency, exposure duration, and work experience. Clinical Biomechanics, 21(4), 345–352. https://doi.org/10.1016/j.clinbiomech.2005.10.004
21.
MeepraditP.SuneeN.ChantrasaR. (2015). The Application of NIOSH Lifting Equation to Prevent Musculoskeletal Disorder Risks. Journal of Biosciences and Medicines, 03(03), 39–44. https://doi.org/10.4236/jbm.2015.33006
22.
MengX.YangY.LiK.YingZ. (2022). A Structure-Aware Adversarial Framework with the Keypoint Biorientation Field for Multiperson Pose Estimation. Wireless Communications and Mobile Computing, 2022.
23.
MutegekiR.HanD. S. (2020, February). A CNN-LSTM approach to human activity recognition. In 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC) (pp. 362-366). IEEE.
24.
SinghR. K.SoniA.KumarS.PasupuletiS.GovindV. (2021). Zonation of flood prone areas by an integrated framework of a hydrodynamic model and ANN. Water Supply, 21(1), 80-97.
25.
SunH.DaiK.ZhaiW.ZhouY.LiJ.ZhengG.LiB.LiuC.ShenC. (2019). A Highly Sensitive and Stretchable Yarn Strain Sensor for Human Motion Tracking Utilizing a Wrinkle-Assisted Crack Structure. ACS Applied Materials & Interfaces, 11(39), 36052–36062. https://doi.org/10.1021/acsami.9b09229
26.
TranJ.UfkesA.FialaM.FerwornA. (2011). Low-cost 3D scene reconstruction for response robots in real-time.2011 IEEE International Symposium on Safety, Security, and Rescue Robotics, 161–166. https://doi.org/10.1109/SSRR.2011.6106780
27.
WangH.XieZ.LuL.LiL.XuX. (2021). A computer-vision method to estimate joint angles and L5/S1 moments during lifting tasks through a single camera. Journal of Biomechanics, 129, 110860.
28.
WangX.HuY. H.LuM. L.RadwinR. G. (2021). Load Asymmetry Angle Estimation Using Multiple-View Videos. IEEE Transactions on Human-Machine Systems, 51(6), 734-739.
29.
WatersT. R.Putz-AndersonV.GargA.FineL. J. (1993). Revised NIOSH equation for the design and evaluation of manual lifting tasks. Ergonomics, 36(7), 749–776. https://doi.org/10.1080/00140139308967940
30.
WeiX.ZhangP.ChaiJ. (2012). Accurate realtime full-body motion capture using a single depth camera. ACM Transactions on Graphics, 31(6), 1–12. https://doi.org/10.1145/2366145.2366207
31.
ZhaoL.ChengX.WuZ. (2009). Study on Image Distortion Caused by Camera Incline in Multi-Format Aerial Digital Camera. 2009 First International Conference on Information Science and Engineering, 1452–1455. https://doi.org/10.1109/ICISE.2009.1123
32.
ZhouS.KeM.LuoP. (2018, July). An Overview of Image-Based Person Re-identification. In International Conference on Applications and Techniques in Cyber Security and Intelligence (pp. 370-378). Springer, Cham.
