Spatial-temporal graph convolutional network for emotion recognition from 3D body gestures on MASR dataset

Abstract

Body movements and gestures can provide cues for emotion expression, which makes 3D skeletal analysis a useful tool for affective computing. This research introduces a lightweight Spatial-Temporal Graph Convolutional Network (ST-GCN)-based architecture for emotion recognition from 3D body movements using the Multimodal Affective State Recognition (MASR) dataset. The human body is represented as a spatio-temporal graph, with skeletal joints functioning as nodes and their spatial and temporal dependencies being modeled through graph-based learning. A preprocessing pipeline is used to prepare the data for model training, includes extracting skeleton sequences, normalizing, temporal alignment and input transformation. To better fit the small size of the MASR dataset, the framework uses a simpler ST-GCN architecture with fewer blocks and optimized channel dimensions. The adapted framework achieves 93.0% accuracy on the MASR dataset, according to experimental results. The results show that lightweight graph-based architectures can effectively recognize emotions from 3D skeletal data by identifying distinct spatio-temporal motion patterns. They also show how important it is to establish a balance between model complexity and generalization when working with small datasets.