Generative deep learning has emerged as a promising data augmentation technique in recent years. This approach becomes particularly valuable in areas such as motion analysis, where it is challenging to collect substantial amounts of data. The objective of the current study is to introduce a data augmentation strategy that relies on a variational autoencoder to generate synthetic data of kinetic and kinematic variables. The kinematic and kinetic variables consist of hip and knee joint angles and moments, respectively, in both sagittal and frontal plane, and ground reaction forces. Statistical parametric mapping (SPM) did not detect significant differences between real and synthetic data for each of the biomechanical variables considered. To further evaluate the effectiveness of this approach, a long-short term model (LSTM) was trained both only on real data (R) and on the combination of real and synthetic data (R&S); the performance of each of these two trained models was then assessed on real test data unseen during training. The principal findings included achieving comparable results in terms of nRMSE when predicting knee joint moments in the frontal (R&S: 9.86% vs R: 10.72%) and sagittal plane (R&S: 9.21% vs R: 9.75%), and hip joint moments in the frontal (R&S: 16.93% vs R: 16.79%) and sagittal plane (R&S: 13.29% vs R: 14.60%). The main novelty of this study lies in introducing an effective data augmentation approach in motion analysis settings.
WangLHuWTanT.Recent developments in human motion analysis. Pattern Recognit2003; 36(3): 585–601.
2.
OrtegaBPOlmedoJM.Application of motion capture technology for sport performance analysis. Retos2017; 32: 241–247.
3.
DevanneM.Multi-level motion analysis for physical exercises assessment in kinaesthetic rehabilitation. In: 2017 IEEE-RAS 17th international conference on humanoid robotics (Humanoids), Birmingham, UK, 15–17November2017, pp.529–534. New York: IEEE.
4.
MenacheA.Understanding motion capture for computer animation and video games. San Diego, CA: Morgan Kaufmann, 2000.
5.
CamomillaVCereattiACuttiAG,et al. Methodological factors affecting joint moments estimation in clinical gait analysis: a systematic review. Biomed Eng Online2017; 16(1): 1–27.
6.
MiyoshiTShirotaTYamamotoS-I,et al. Effect of the walking speed to the lower limb joint angular displacements, joint moments and ground reaction forces during walking in water. Disabil Rehabil2004; 26(12): 724–732.
7.
LiuRQianDChenY,et al. Investigation of normal knees kinematics in walking and running at different speeds using a portable motion analysis system. Sports Biomech2024; 23(4): 417–430.
8.
PollardCDSigwardSMPowersCM.Gender differences in hip joint kinematics and kinetics during side-step cutting maneuver. Clin J Sport Med2007; 17(1): 38–42.
9.
HartJMKoJ-WKKonoldT,et al. Sagittal plane knee joint moments following anterior cruciate ligament injury and reconstruction: a systematic review. Clin Biomech2010; 25(4): 277–283.
10.
ConstantinouMLoureiroACartyC,et al. Hip joint mechanics during walking in individuals with mild-to-moderate hip osteoarthritis. Gait Posture2017; 53: 162–167.
11.
CvetanovichGLFarkasGJBeckEC,et al. Squat and gait biomechanics 6 months following hip arthroscopy for femoroacetabular impingement syndrome. J Hip Preserv Surg2020; 7(1): 27–37.
12.
OrnettiPMaillefertJFLarocheD,et al. Gait analysis as a quantifiable outcome measure in hip or knee osteoarthritis: a systematic review. Joint Bone Spine2010; 77(5): 421–425.
13.
van EgmondNStolwijkNvan HeerwaardenR,et al. Gait analysis before and after corrective osteotomy in patients with knee osteoarthritis and a valgus deformity. Knee Surg Sports Traumatol Arthrosc2017; 25: 2904–2913.
14.
PerroneMGuidettiMGalliM,et al. Hip joint contact forces are lower in people with femoroacetabular impingement syndrome during squat tasks. J Orthop Res2024; 42(5): 1045–1053.
15.
LowWSChanCKChuahJH,et al. A review of machine learning network in human motion biomechanics. J Grid Comput2022; 20(1): 4.
16.
KheraPKumarN.Role of machine learning in gait analysis: a review. J Med Eng Technol2020; 44(8): 441–467.
17.
MouloodiSRahmanpanahHGohariS,et al. What can artificial intelligence and machine learning tell us? A review of applications to equine biomechanical research. J Mech Behav Biomed Mater2021; 123: 104728.
18.
RaneLDingZMcGregorAH,et al. Deep learning for musculoskeletal force prediction. Ann Biomed Eng2019; 47: 778–789.
19.
MolinaroDDKangICamargoJ,et al. Subject-independent, biological hip moment estimation during multimodal overground ambulation using deep learning. IEEE Trans Med Robot Bionics2022; 4(1): 219–229.
20.
HalilajERajagopalAFiterauM,et al. Machine learning in human movement biomechanics: best practices, common pitfalls, and new opportunities. J Biomech2018; 81: 1–11.
21.
GurchiekRDCheneyNMcGinnisRS.Estimating biomechanical time-series with wearable sensors: a systematic review of machine learning techniques. Sensors2019; 19(23): 5227.
22.
StetterBJRinghofSKrafftFC,et al. Estimation of knee joint forces in sport movements using wearable sensors and machine learning. Sensors2019; 19(17): 3690.
23.
SitoleSPSupFCIV. Continuous prediction of human joint mechanics using EMG signals: a review of model-based and model-free approaches. IEEE Trans Med Robot Bionics2023; 5(3): 528–546.
24.
HaratoKIwamaYNishizawaK,et al. Estimation of knee adduction moment using a single inertial measurement unit and deep learning. Osteoarthritis Cartilage2022; 30: S92.
25.
MundtMKoeppeADavidS,et al. Estimation of gait mechanics based on simulated and measured IMU data using an artificial neural network. Front Bioeng Biotechnol2020; 8: 41.
26.
Sharifi RenaniMEustaceAMMyersCA,et al. The use of synthetic IMU signals in the training of deep learning models significantly improves the accuracy of joint kinematic predictions. Sensors2021; 21(17): 5876.
RadfordAWuJChildR,et al. Language models are unsupervised multitask learners. OpenAI blog2019; 1(8): 9.
29.
AkkemYKumarBBiswasSK,et al. Streamlit application for advanced ensemble learning methods in crop recommendation systems–a review and implementation. Indian J Sci Technol2023; 16(48): 4688–4702.
30.
AkkemYKumarBBiswasSK,et al. Streamlit-based enhancing crop recommendation systems with advanced explainable artificial intelligence for smart farming. Neural Comput Appl2024; 36(32): 20011–20025.
31.
AkkemYKumarBBiswasSK,et al. Smart farming using artificial intelligence: a review. Eng Appl Artif Intell2023; 120: 105899.
32.
AkkemYKumarBBiswasSK,et al. A comprehensive review of synthetic data generation in smart farming by using variational autoencoder and generative adversarial network. Eng Appl Artif Intell2024; 131: 107881.
33.
WieseMKnoblochRKornR,et al. Quant GANs: deep generation of financial time series. Quant Financ2020; 20(9): 1419–1444.
34.
LinZShiYXueZ,et al. IDSGAN: generative adversarial networks for attack generation against intrusion detection. In: Pacific-Asia conference on knowledge discovery and data mining, pp.79–91. Cham: Springer, 2022.
35.
BicerMPhillipsATMelisA,et al. Generative deep learning applied to biomechanics: a new augmentation technique for motion capture datasets. J Biomech2022; 144: 111301.
36.
WangYLiZWangX,et al. Human gait data augmentation and trajectory prediction for lower-limb rehabilitation robot control using GANs and attention mechanism. Machines2021; 9(12): 367.
PerroneMMellSPMartinJ,et al. Machine learning-based prediction of hip joint moment in healthy subjects, patients and post-operative subjects. Comput Methods Biomech Biomed Engin2024: 1–5.
39.
MalloyPWichmanDMGarciaF,et al. Impaired lower extremity biomechanics, hip external rotation muscle weakness, and proximal femoral morphology predict impaired single-leg squat performance in people with FAI syndrome. Am J Sports Med2021; 49(11): 2984–2993.
40.
MalloyPNeumannDAKippK,et al. Hip biomechanics during a single-leg squat: 5 key differences between people with femoroacetabular impingement syndrome and those without hip pain. J Orthop Sports Phys Ther2019; 49(12): 908–916.
41.
McGovernRPChristoforettiJJMartinRL,et al. Evidence for reliability and validity of functional performance testing in the evaluation of nonarthritic hip pain. J Athl Train2019; 54(3): 276–282.
42.
McGovernRPMartinRLPhelpsAL,et al. Conservative management acutely improves functional movement and clinical outcomes in patients with pre-arthritic hip pain. J Hip Preserv Surg2020; 7(1): 95–102.
43.
MartinRLTaklaADisantisA,et al. Evaluating functional performance tests in those with non-arthritic intra-articular hip pain: an international consensus statement. Int J Sports Phys Ther2023; 18(6): 1346.
44.
YoonJJarrettDVan der SchaarM.Time-series generative adversarial networks. Adv Neural Inf Process Syst2019; 32.
45.
DesaiAFreemanCWangZ,et al. Timevae: a variational auto-encoder for multivariate time series generation. arXiv preprint arXiv:2111.08095, 2021.
46.
JinX-BGongW-TKongJ-L,et al. PFVAE: a planar flow-based variational auto-encoder prediction model for time series data. Mathematics2022; 10(4): 610.
MaedaTUkitaN.Motionaug: augmentation with physical correction for human motion prediction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2022, pp.6427–6436. IEEE Xplore.
52.
MundtMJohnsonWRPotthastW,et al. A comparison of three neural network approaches for estimating joint angles and moments from inertial measurement units. Sensors2021; 21(13): 4535.
53.
McCabeMVVan CittersDWChapmanRM.Developing a method for quantifying hip joint angles and moments during walking using neural networks and wearables. Comput Methods Biomech Biomed Engin2023; 26(1): 1–11.
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
