Automatic choreography and curriculum resource integration based on improved LSTM with multimodal entity linking

Abstract

With the development of artificial intelligence technology, automatic choreography, as an emerging cross-disciplinary field, has received more and more attention in its research and application. To realize automatic choreography as well as the storage of dance curriculum resources, the study first introduces the bidirectional long short-term memory model, and combines the attention mechanism for music generation. The attention mechanism enables the model to focus more on important feature information by assigning different weights to different time steps in the model, thereby better understanding the overall information of the music sequence and improving the performance of the model. Then, the Openpose model is utilized for human pose estimation and the sequence-to-sequence model is used to generate dance movements matching the music. The experimental results show that the Att-BiLSTM model outperforms the traditional model in terms of accuracy, recall rate, precision value and F1 value. Compared with the traditional LSTM model, the accuracy of the Att-BiLSTM model has increased from 85.2% to 94.9%, the recall rate from 79.1% to 90.3%, the precision value from 87.5% to 95.8%, and the F1 value from 84.7% to 94.7%. The performance has improved significantly. It reflects the significant improvement effect of the attention mechanism on the performance of the BiLSTM model. In terms of human pose estimation, the Openpose model keypoint detection accuracy and partial affinity field prediction effect reached 0.927 and 0.854, respectively, and the frame rate reached 15FPS. The Seq2Seq network was located at the highest level in terms of movement flow, naturalness, and synchronization scores in dance movement generation. The movement coherence index and music rhythm matching were 0.92 and 0.95, respectively. The results demonstrate that the network model proposed in the study has significant advantages in terms of coherence, naturalness, and synchronization with music in movement generation. This is of great practical significance for promoting the development of automatic choreography technology and the innovation of dance education.