Emotional responses to visual art involve perceptual and physiological interactions, but existing methods isolate these modalities, limiting individualized affective modeling. This study presents BioArt-Net, a computational framework integrating visual semantic analysis and physiological signal processing for viewer-centric art emotion recognition, focusing on cross-modal fusion methodologies. Key modules: Visual semantics via fine-tuned ViT (16 × 16 patches, 32% dimensionality reduction); EEG (wavelet transform), GSR/PPG (1D CNNs) encoding with 94% data integrity; token-level attention aligning modalities; compound loss (CE + MSE + entropy) reducing redundancy by 27%; optimized via distillation (48% fewer parameters, 89 ms latency). On 520-session BioArt-Emotion Dataset, accuracy reaches 0.87 (vs 0.76 unimodal), F1 = 0.85. Cross-modal attention boosts accuracy by 11%; physiological reweighting stabilizes results. This advances computational neuroaesthetics, fitting the journal’s focus on innovative interdisciplinary frameworks.
WangJZZhaoSWuC, et al.Unlocking the emotional world of visual media: an overview of the science, research, and impact of understanding emotion. Proc IEEE2023; 111(10): 1236–1286.
2.
GreenM. Affective expression in the visual arts. In: The expression of emotion in the visual arts. Routledge, 2024, pp. 46–66.
3.
ChowdhuryJHLiuQRamannaS. Simple histogram equalization technique improves performance of VGG models on facial emotion recognition datasets. Algorithms2024; 17(6): 238.
4.
RonickomJFA. Enhancing emotion recognition: machine learning with phasic spectrogram texture features. 2023 IEEE 5th International Conference on Cybernetics, Cognition and Machine Learning Applications (ICCCMLA). Hamburg, Germany: IEEE, 2023, pp. 600–603.
5.
YamamotoRTakemuraN. EMMA: emotion mixing algorithm for compound expression recognition using angle-based metric learning. In Proceedings of the Asian Conference on Computer Vision (ACCV 2024) Workshops. Hanoi, Vietnam: Springer, 2024, pp. 495–510.
6.
ChenB. Classification of artistic styles of Chinese art paintings based on the CNN model. Comput Intell Neurosci2022; 2022(1): 4520913.
7.
YuQShiC. An image classification approach for painting using improved convolutional neural algorithm. Soft Comput2024; 28(1): 847–873.
8.
GamageTAKalansooriyaLPSandamaliERC. An emotion classification model for driver emotion recognition using electroencephalography (EEG). 2022 international research conference on Smart Computing and Systems Engineering (SCSE). Colombo, Sri Lanka: IEEE, 2022, 5, pp. 76–82.
9.
IsmailSNMSAzizNAAIbrahimSZ. A comparison of emotion recognition system using electrocardiogram (ECG) and photoplethysmogram (PPG). J King Saud Univ-Comput Inf Sci2022; 34(6): 3539–3558.
10.
PatilVKPawarVRRandiveS, et al.From face detection to emotion recognition on the framework of raspberry pi and galvanic skin response sensor for visual and physiological biosignals. J Elect Syst Inf Technol2023; 10(1): 24.
11.
BansalMGoyalAChoudharyA. A comparative analysis of K-nearest neighbor, genetic, support vector machine, decision tree, and long short term memory algorithms in machine learning. Decision Analytics J2022; 3: 100071.
12.
BouaziziSbenmohamedELtifiH. Decision-making based on an improved visual analytics approach for emotion prediction. Intell Decis Technol2023; 17(2): 557–576.
13.
HwooiSKWOthmaniASabriAQM. Deep learning-based approach for continuous affect prediction from facial expression images in valence-arousal space. IEEE Access2022; 10: 96053–96065.
14.
JoudehIOCretuAMBouchardS. Predicting the arousal and valence values of emotional states using learned, predesigned, and deep visual features. Sensors2024; 24(13): 4398.
15.
YangHChenJ. Art appreciation model design based on improved PageRank and ECA-ResNeXt50 algorithm. PeerJ Comput Sci2023; 9: e1734.
16.
AmraniGAdadiABerradaM. An attention mechanism-based interpretable model for epileptic seizure detection and localization with self-supervised pre-training. IEEE Access2025; 16: 60213–60232.
17.
LinWLiC. Review of studies on emotion recognition and judgment based on physiological signals. Appl Sci2023; 13(4): 2573.
18.
RamaswamyMPAPalaniswamyS. Multimodal emotion recognition: a comprehensive review, trends, and challenges. WIREs Data Min & Knowl2024; 14(6): e1563.
19.
ArslanEEAkşahinMFYilmazM, et al.Towards emotionally intelligent virtual environments: classifying emotions through a biosignal-based approach. Appl Sci2024; 14(19): 8769.
20.
ShiYZhengXZhangM, et al.A study of subliminal emotion classification based on entropy features. Front Psychol2022; 13: 781448.
21.
Almanza-ConejoOAlmanza-OjedaDLContreras-HernandezJL, et al.Emotion recognition in EEG signals using the continuous wavelet transform and CNNs. Neural Comput Appl2023; 35(2): 1409–1422.
22.
SiamAIEl-ShafaiWAbou ElazmLA, et al.Enhanced user verification in IoT applications: a fusion-based multimodal cancelable biometric system with ECG and PPG signals. Neural Comput Appl2024; 36(12): 6575–6595.
23.
WuTJiangNKumarTBJ, et al.The role of cognitive factors in consumers’ perceived value and subscription intention of video streaming platforms: a systematic literature review. Cogent Bus Manag2024; 11(1): 2329247.
24.
MoSMorgadoP. Unveiling the power of audio-visual early fusion transformers with dense interactions through masked modeling. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. Seattle, Washington, USA: IEEE, 2024, pp. 27186–27196.
25.
ZhuLZhuZZhangC, et al.Multimodal sentiment analysis based on fusion methods: a survey. Inf Fusion2023; 95: 306–325.
26.
GhaithS. Deep context transformer: bridging efficiency and contextual understanding of transformer models. Appl Intell2024; 54(19): 8902–8923.
27.
AloiDBouzitSLiJ. Biometric methods for user research: three case studies. In: Extended abstracts of the CHI conference on human factors in computing systems, 2024, pp. 1–8.
28.
YaoLLuYQianY, et al.High-accuracy classification of multiple distinct human emotions using EEG differential entropy features and ResNet18. Appl Sci2024; 14(14): 6175.
29.
HelalyRMessaoudSBouaafiaS, et al.DTL-I-ResNet18: facial emotion recognition based on deep transfer learning and improved ResNet18. Signal Image Video Process2023; 17(6): 2731–2744.
30.
BanikSKumarHGanapathyN, et al.Exploring central-peripheral nervous system interaction through multimodal biosignals: a systematic review. IEEE Access2024.
31.
QiaoYXuHMZhouWJ, et al.A BiGRU joint optimized attention network for recognition of drilling conditions. Pet Sci2023; 20(6): 3624–3637.
32.
XuLXuWCuiQ, et al.Deep heuristic evolutionary regression model based on the fusion of BiGRU and BiLSTM. Cognit Comput2023; 15(5): 1672–1686.
33.
HungBTThuNHM. Novelty fused image and text models based on deep neural network and transformer for multimodal sentiment analysis. Multimed Tool Appl2024; 83(25): 66263–66281.
34.
WangYHeJWangD, et al.Multimodal transformer with adaptive modality weighting for multimodal sentiment analysis. Neurocomputing2024; 572: 127181.
