Sage Journals: Discover world-class research

Abstract

The effectiveness of online music education relies heavily on understanding and addressing students’ emotional states, which can impact engagement and learning outcomes. This paper presents a novel emotion recognition method based on an improved frame attention network (IFAN), designed specifically for online music teaching. The method utilizes facial expression data to identify four key emotional states—pleasure, concentration, confusion, and boredom—by introducing deformable convolution to better capture dynamic facial features and a feature aggregation module to enhance emotional temporal patterns. The proposed model achieves recognition accuracies of 96%, 94%, 93%, and 98% for each emotional state, outperforming existing emotion recognition methods. Experimental results indicate that the model is robust and highly accurate in the context of online music education. This research provides a foundation for real-time emotion recognition systems in online teaching environments, with potential for future work to incorporate multimodal data, such as audio and physiological signals, to further enhance model performance.

Keywords

music teaching online learning emotional recognition frame attention network

Get full access to this article

View all access options for this article.

References

Huang

, et al. Research and implementation of data-driven online learning burnout early warning model. E-education Res 2021; 42(2): 47–54.

Guo

. Solutions to the lack of teaching emotion between teachers and students in online teaching under the epidemic situation. The Chinese J ICT in Educ 2021; (7): 65–69.

Dudan Studiawan

Ferrous

Payne

. Anomaly detection in operating system logs with deep learning-based sentiment analysis. IEEE Trans Dependable Secure Comput 2021; 18(5): 2136.

Shevchenko

Makarov

. Neural network model for video-based analysis of student’s emotions in E-learning. Opt Mem Neural Network 2022; 31(3): 237–244.

Cao

Zhang

Pan

, et al. Research on recognition method of learning participation from the perspective of artificial intelligence analysis of deep learning experiment based on multimodal data fusion. J Distance Educ 2019; (1): 32–44.

Tao

Chen

. Research on emotion recognition based on eye movement features in online learning environment. Comput Technol Dev 2021; 31(3): 186–190.

Zhao

. Emotion recognition and education. Artif Intell 2019; (3): 29–36.

Pise

Vadapall

Sanders

. Facial emotion recognition using temporal relational network: an application to E-learning. Multimed Tool Appl 2020; 81(2): 1–12.

Dinar

Jarvela

Ahola

, et al. Leaders and followers identified by emotional mimicry during collaborative learning: a facial expression recognition study on emotional valence. IEEE Trans Affect Comput 2020; 13(3): 1390–1400.

10.

Park

Ryu

. Exploring preservice teachers’ emotional experiences in an immersive virtual teaching simulation through facial expression recognition. Int J Hum Comput Interact 2019; 35(6): 521–533.

11.

Ashwin

Ram

MRG

. Automatic detection of student affective states in classroom environment using hybrid convolutional neural networks. Educ Inf Technol 2020; 25(2): 1387–1415.

12.

. Emotion recognition of learning picture in intelligent learning environment and its application. Jinan: ShandongNormal University, 2019.

13.

. Design and implementation of emotion analysis model for online learning based on multimodality. Nanjing: Nanjing Normal University, 2019.

14.

Jia

Zhang

Zhao

, et al. Classroom student state analysis based on artificial intelligence video processing. Mod Educ Technol 2019; 29(12): 82–88.

15.

Chen

Cheng

Song

, et al. Video based heart rate measurement; recent advances and future prospects. IEEE Trans Instrum Meas 2018; 68(10): 3600–3615.

16.

Estrada

MLB

Cabada

Bustillos

, et al. Opinion mining and emotion recognition applied to learning environments. Expert Syst Appl 2020; 150(7): 12–23.

17.

Cabada

Estrada

MLB

Bustillos

. Mining of educational opinions with deep learning. J Univers Comput Sci 2018; (11): 1604–1626.

18.

Balalahadia

Fernando

Juanatas

. Teacher’s performance evaluation tool using opinion mining with sentiment analysis. In: 2016 IEEE Region 10 Symposium (TENSYMP), Bali, Indonesia, 09–11 May 2016.

19.

Shenoy

Sardana

. Multilogue-net: a context-aware RNN for multi-modal emotion detection and sentiment analysis in conversation. arXiv preprint arXiv: 2002.08267 2020.

20.

Zhang

Ren

, et al. Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778. Las Vegas, NV, USA.

21.

Meng

Peng

Wang

, et al. Frame attention networks for facial expression recognition in videos. In: 2019 IEEE international conference on image processing(ICIP), 2019, pp. 3866–3870. Taipei, Taiwan.

22.

Wei

. Research on non-contact heart rate detection based on face video. Chengdu: Xihua University, 2022.

23.

Guo

Zhang

, et al. MS celeb lM: a dataset and benchmark for large scale face recognition. In: European conference on computer vision. Cham: Springer, 2016, pp. 87–102.

24.

Barsoum

Zhang

Ferrer

, et al. Training deep networks for facial expression recognition with crowd-sourced label distribution. In: Proceedings of the 18th ACM international conference on multimodal interaction, 2016, pp. 279–283. New York, NY, United States.

25.

Yue

Liu

. Improved FunkSVD algorithm based on RMSProp. J Circ Syst Comput 2022; 31(08): 2250139.

26.

Ignacio

Castaneda

Marin

, et al. Correction: what do we want to know about MOOCs? Results from a machine learning approach to a systematic literature mapping review. Int J Educ Technol High Educ 2022; 19(1): 63.

Analysis of students’ emotion based on visual clues in online music teaching

Abstract

Keywords

Get full access to this article

References