Sage Journals: Discover world-class research

Abstract

Addressing the challenges of spatio-temporal continuity and ergonomic optimization in dynamic action generation for theatrical performance roles, this study introduces a novel generative adversarial network (GAN)-driven framework. We construct a spatio-temporal constraint generator utilizing the Archive of Motion Capture as Surface Shapes (AMASS) motion capture dataset, integrated with a biomechanical feedback mechanism to optimize joint load distribution. Experimental validation demonstrates that our method significantly enhances action naturalness in choreographed fight and prop-handling scenarios, achieving an Fréchet Inception Distance (FID) of 29.7 (16.6% lower than the optimal baseline PhysGAN). Furthermore, it reduces peak lumbar spine loads by 24.2% and optimizes rapid entire body assessment (REBA) scores by 21.5% compared to PhysGAN (from 6.5 to 5.1). The proposed framework delivers a high-fidelity, low-risk action generation solution for automated performer-technology collaboration systems, enhancing performance realism and actor safety with demonstrable theatrical/cinematic application value.

Keywords

GANs dynamic action generation biomechanical optimization ergonomics evaluation AMASS database

Get full access to this article

View all access options for this article.

References

Tseng

C-J

Abdalla

. A human–computer system for collaborative design (HCSCD). J Mater Process Technol 2004; 155: 1964–1971.

Silver

. Multi-dimensional pre-programmed and learned fine motor control for a humanoid robot. University of Illinois at Urbana-Champaign, 2012, Vol. 1, p. 15.

, et al. Global burden of occupational ergonomic factor-induced low back pain, 1990∼ 2021: data analysis and projections of the global burden of disease. Front Public Health 2025; 13: 1573828.

Moeslund

Hilton

Krüger

. A survey of advances in vision-based human motion capture and analysis. Comput Vis Image Understand 2006; 104: 90–126.

Tejera

Casas

Hilton

. Animation control of surface motion capture. IEEE Trans Cybern 2013; 43: 1532–1545.

Peng

Abbeel

Levine

, et al. Deepmimic: example-guided deep reinforcement learning of physics-based character skills. ACM Trans Graph 2018; 37: 1–14.

Won

Gopinath

Hodgins

. Control strategies for physically simulated characters performing two-player competitive sports. ACM Trans Graph 2021; 40: 1–11.

Holden

Saito

Komura

. A deep learning framework for character motion synthesis and editing[J]. ACM Transactions on Graphics (ToG) 2016; 35(4): 1–11.

Zhao

Tang

Xie

, et al. Bidirectional transformer gan for long-term human motion prediction. ACM Trans Multimed Comput Commun Appl 2023; 19: 1–19.

10.

Jin

Wang

, et al. Exploiting attention-consistency loss for spatial-temporal stream action recognition. ACM Trans Multimed Comput Commun Appl 2022; 18: 1–15.

11.

Hilmi

Hamid

ARA

Waraw

. Recent advancements in ergonomic risk assessment: integration of artificial intelligence, wearable technology, and industry-specific approaches. MJEr 2024; 6: 65–75.

12.

Shu

Zhang

G-J

, et al. Spatiotemporal co-attention recurrent neural networks for human-skeleton motion prediction. IEEE Trans Pattern Anal Mach Intell 2021; 44: 3300–3315.

13.

David

. Ergonomic methods for assessing exposure to risk factors for work-related musculoskeletal disorders. Occup Med 2005; 55: 190–199.

14.

Jia

. Human motion modeling with deep learning: a survey. AI Open 2022; 3: 35–39.

15.

Ionescu

Papava

Olaru

, et al. Human3. 6m: large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Trans Pattern Anal Mach Intell 2013; 36: 1325–1339.

16.

Mourot

Hoyet

Le Clerc

, et al. A survey on deep learning for skeleton-based human animation. Comput Graph Forum 2022; 41: 122–157.

17.

Wang

Gou

Duan

, et al. Generative adversarial networks: introduction and outlook. IEEE/CAA J Autom Sinica 2017; 4: 588–598.

18.

Chen

. A spatiotemporal motion prediction network based on multi-level feature disentanglement. Image Vis Comput 2024; 146: 105005.

19.

Saito

Koyama

, et al. Train sparsely, generate densely: Memory-efficient unsupervised training of high-resolution temporal gan. Int J Comput Vis 2020; 128: 2586–2606.

20.

Wang

Chai

Xia

. Combining recurrent neural networks and adversarial training for human motion synthesis and control. IEEE Trans Vis Comput Graph 2019; 27: 14–28.

21.

Jin

Huang

, et al. Driving behavioral learning leveraging sensing information from innovation hub. Center for Advanced Infrastructure and Transportation, 2022, Vol. 1, pp. 7–89.

22.

Vakanski

. Generative adversarial networks for generation and classification of physical rehabilitation movement episodes. Int J Mach Learn Comput 2018; 8: 428–436.

23.

Seifert

Komar

Araújo

, et al. Neurobiological degeneracy: a key property for functional adaptations of perception and action to constraints. Neurosci Biobehav Rev 2016; 69: 159–165.

24.

Veikko

. OWAS: a method for the evaluation of postural load during work. Topeliuksenkatu 1992; 1: 2–20.

25.

Hignett

McAtamney

. Rapid entire body assessment (REBA). Appl Ergon 2000; 31: 201–205.

26.

Bicer

Phillips

Melis

, et al. Generative deep learning applied to biomechanics: a new augmentation technique for motion capture datasets. J Biomech 2022; 144: 111301.

27.

Baltzopoulos

. Inverse dynamics, joint reaction forces and loading in the musculoskeletal system: guidelines for correct mechanical terms and recommendations for accurate reporting of results. Sports Biomech 2024; 23: 287–300.

28.

Dindorf

Dully

Konradi

, et al. Enhancing biomechanical machine learning with limited data: generating realistic synthetic posture data using generative artificial intelligence. Front Bioeng Biotechnol 2024; 12: 1350135.

Generative adversarial network-driven dynamic motion generation and optimization for labor roles

Abstract

Keywords

Get full access to this article

References