Sage Journals: Discover world-class research

Abstract

Real-time emotion detection is a crucial aspect of media, allowing for empathetic and tailored responses to users’ emotions. Customized interaction analyzing data to understand individual user needs and adjust interactions accordingly. Real-time emotion detection can face accuracy issues due to varying individual expressions and contexts, while customized interaction analysis can be limited by data privacy concerns and data integration challenges. The research introduces a novel real-time emotion detection model named E-IS to monitor the emotional changes and interactions of users when viewing broadcast hosting services. This service provides live or pre-recorded content to visitors through various platforms. Early on, audience facial expressions and interaction records are collected to detect audience emotions and analyze personalized interactions. The image resizing, noise removal, removal of stop words, stemming, and emoji and symbol handling processes are employed for data pre-processing. The EN-CNN is for emotion detection, and AIDT is for analyzing customized interaction using the normalized data. These prediction models are deployed in E-IS using Python. E-IS integrates with the broadcast hosting communication network hardware and software components. The results demonstrate that the E-IS achieved the highest F1-score (96.70%), recall (95%), precision (97%), accuracy (99%), user satisfaction score (95%), and engagement level (92%). The E-IS enhances broadcast hosting services. Additionally, it effectively reduced the emotion detection latency, response time for interaction adjustment, false positive rate in emotion detection, and data processing time per interaction.

Keywords

emotion detection customized interaction broadcast hosting Emo-Interact system communication network Adaptive Interaction Decision Tree (AIDT)EmoNet-Convolutional neural network (EN-CNN)

Get full access to this article

View all access options for this article.

References

Kumar

Govarthan

Gadda

, et al. Deep learning-based automated emotion recognition using multi modal physiological signals and time-frequency methods. IEEE Trans Instrum Meas 2024; 73: 1–12. DOI: 10.1109/TIM.2024.3420349.

Peng

. End-to-end multimodal emotion recognition based on facial expressions and remote photoplethysmography signals. IEEE J Biomed Health Inform 2024; 28: 6054–6063. DOI: 10.1109/JBHI.2024.3430310.

Blanco-Ríos

Candela-Leal

Orozco-Romo

, et al. Real-time EEG-based emotion recognition for neurohumanities: perspectives from principal component analysis and tree-based algorithms. Front Hum Neurosci 2024; 18: 1319574. DOI: 10.3389/fnhum.2024.1319574.

Rokhsaritalemi

Sadeghi-Niaraki

Choi

. Exploring emotion analysis using artificial intelligence, geospatial information systems, and extended reality for urban services. IEEE Access 2023; 11: 92478–92495. DOI: 10.1109/ACCESS.2023.3307639.

Maroto-Gómez

Castro-González

Castillo

, et al. An adaptive decision-making system supported by user preference predictions for human-robot interactive communication. User Model User-adapt Interact 2023; 33(2): 359–403. DOI: 10.1007/s11257-022-09321-2.

Gonulal

. Investigating EFL learners’ humorous interactions with an intelligent personal assistant. Interact Learn Environ 2023; 31(7): 4521–4534. DOI: 10.1080/10494820.2021.1974489.

Dalabehera

Bebortta

Singh

, et al. Real-time criticality evaluation through vision-based human-centric emotion, activity, and object interactions. 2024 1st International Conference on Cognitive, Green and Ubiquitous Computing (IC-CGU). Piscataway, NJ: IEEE, 2024, pp. 1–6. DOI: 10.1109/IC-CGU58078.2024.10530702.

Jyotsna

Amudha

Ram

, et al. PredictEYE: Personalized time series model for mental state prediction using eye tracking. IEEE Access 2023; 11: 128383–128409. DOI: 10.1109/ACCESS.2023.3332762.

Karilingappa

Jayadevappa

Ganganna

. Human emotion detection and classification using modified Viola-Jones and convolution neural network. IAES Int J Artif Intell 2023; 12(1): 79. DOI: 10.11591/ijai.v12.i1.pp79-86.

10.

Yan

Lin

, et al. A novel exploitative and explorative GWO-SVM algorithm for smart emotion recognition. IEEE Internet Things J 2023; 10(11): 9999–10011. DOI: 10.1109/JIOT.2023.3235356.

11.

Sharma

Bajaj

Arora

. Machine learning techniques for real-time emotion detection from facial expressions. 2023 of IEEE Delhi Section Flagship Conference (DELCON). 2nd ed. Piscataway, NJ: IEEE, 2023, pp. 1–6. DOI: 10.1109/DELCON57910.2023.10127369.

12.

Gan

Zheng

Zhu

, et al. A graph neural network with context filtering and feature correction for conversational emotion recognition. Inf Sci 2024; 658: 120017. DOI: 10.1016/j.ins.2023.120017.

13.

Chen

Garg

, et al. Graph enhanced low-resource ECG representation learning for emotion recognition based on wearable internet of things. IEEE Internet Things J 2024; 1. DOI: 10.1109/JIOT.2024.3430297.

14.

Gilakjani

Al Osman

. A graph neural network for EEG-based emotion recognition with contrastive learning and generative adversarial neural network data augmentation. IEEE Access 2023; 12: 113–130. DOI: 10.1109/ACCESS.2023.3344476.

15.

Aslan

. CNN-based efficient approach for emotion recognition. J King Saud Univ - Comput Inf Sci 2022; 34(9): 7335–7346. DOI: 10.1016/j.jksuci.2021.08.021.

16.

Gupta

Kumar

Tekchandani

. Facial emotion recognition based real-time learner engagement detection system in online learning context using deep learning models. Multimed Tool Appl 2023; 82(8): 11365–11394. DOI: 10.1007/s11042-022-13558-9.

17.

Dixit

Satapathy

. Deep CNN with late fusion for real-time multimodal emotion recognition. Expert Syst Appl 2024; 240: 122579. DOI: 10.1016/j.eswa.2023.122579.

18.

Talaat

Ali

Mostafa

, et al. Real-time facial emotion recognition model based on kernel autoencoder and convolutional neural network for autism children. Soft Comput 2024; 28: 1–4. DOI: 10.1007/s00500-023-09477-y.

19.

Saadon

Yang

Burgert

, et al. Real-time emotion detection by quantitative facial motion analysis. PLoS One 2023; 18(3): e0282730. DOI: 10.1371/journal.pone.0282730.

20.

Jaiswal

Nandi

. Optimized, robust, real-time emotion prediction for human-robot interactions using deep learning. Multimed Tool Appl 2023; 82(4): 5495–5519. DOI: 10.1007/s11042-022-12794-3.

21.

Manalu

Rifai

. Detection of human emotions through facial expressions using hybrid convolutional neural network-recurrent neural network algorithm. Intell Syst Appl 2024; 21: 200339. DOI: 10.1016/j.iswa.2024.200339.

22.

Prince

Babu

. Advancing multiclass emotion recognition with CNN-RNN architecture and illuminating module for real-time precision using facial expressions. 2024 international conference on Advances in Modern Age Technologies for Health and Engineering Science (AMATHE). Piscataway, NJ: IEEE, 2024, pp. 1–11. DOI: 10.1109/AMATHE61652.2024.10582058.

23.

Ashok Kumar

Maddala

Martin Sagayam

. Enhanced facial emotion recognition by optimal descriptor selection with a neural network. IETE J Res 2023; 69(5): 2595–2614. DOI: 10.1080/03772063.2021.1902868.

24.

Sathya

Sudha

. An adaptive fuzzy ensemble model for facial expression recognition using poplar optimization and CRNN. IETE J Res 2023; 8: 1–2. DOI: 10.1080/03772063.2023.2220691.

25.

Thuseethan

Rajasegarar

Yearwood

. Deep3DCANN: a Deep 3DCNN-ANN framework for spontaneous micro-expression recognition. Inf Sci 2023; 630: 341–355. DOI: 10.1016/j.ins.2022.11.113.

26.

Al-Abboodi

Al-Ani

. A novel technique for facial recognition based on the GSO-CNN deep learning algorithm. J Electr Comput Eng 2024; 2024(1): 3443028–3443121. DOI: 10.1155/2024/3443028.

Real-time emotion detection and customized interaction analysis in broadcast hosting

Abstract

Keywords

Get full access to this article

References