Fusion of AI-Based Sequential Models and Optimization Algorithms for Motion Capture Analysis of Cultural Dances

Abstract

The preservation of intangible cultural heritage requires computational frameworks capable of reconstructing traditional dance movements with both numerical accuracy and natural expressiveness. This study introduces a fusion methodology that integrates Principal Component Analysis (PCA) for dimensionality reduction, Hidden Markov Models (HMMs) and their variants for sequential modeling, and a Genetic Algorithm (GA) for family-aware hyperparameter optimization. Skeleton data from the Bedoyo Majapahit dance were captured using markerless motion capture, producing 3341 frames and 33 joints (99 features). PCA reduced the features to 30 principal components, retaining ≈99% of the variance. The proposed framework includes two novel elements: (1) Expected-Centroid decoding for Multinomial HMMs to eliminate stair-step artifacts, and (2) a normalized tri-metric fitness function combining Mean Squared Error (MSE), Dynamic Time Warping (DTW), and Fréchet distance. Experimental results demonstrate that the Hybrid GA–GMM-HMM with eight states and 25 mixtures achieved superior performance (MSE ≈ 0.80, DTW ≈ 1150.3, Fréchet ≈ 2.63), outperforming Gaussian and Multinomial baselines. Visualization of PC1 overlays and PC1–PC2 trajectories further confirmed the proximity of generated sequences to real data. These findings underline the potential of feature-level and model-level fusion for digital documentation and interactive applications in cultural heritage.