Sage Journals: Discover world-class research

Abstract

Digital Chinese landscape painting can be enhanced through multisensory fusion, yet culturally adapted cross-modal mappings and real-time adaptation remain under-tested. This laboratory, between-subjects virtual reality experiment (N = 80) compared a visual-only condition with a multisensory condition delivering synchronized auditory, haptic, and olfactory cues, with cue mapping implemented as culturally congruent or deliberately incongruent. Outcomes included standardized immersion/presence measures and affect ratings, behavioral logs of interaction and exploration (including time-on-task and coverage), and physiological indices of arousal derived from electrodermal activity and photoplethysmography-based pulse metrics, assessed immediately after exposure and at one-week and one-month follow-ups. Culturally congruent multisensory cueing produced higher immersion/presence and engagement than visual-only presentation and the incongruent multisensory format, alongside more frequent and diverse interaction behavior, longer time-on-task, and broader exploration of the virtual scenes. Self-reported emotion showed higher positive valence and arousal under congruent cueing with convergent physiological arousal patterns, and differences remained observable at follow-up assessments.

Keywords

virtual reality user-Computer interface emotions electrodermal activity heart rate olfaction

Get full access to this article

View all access options for this article.

References

Alhakamy

A. A.

(2024). Extended reality (XR) toward building immersive solutions: The key to unlocking industry 4.0. ACM Computing Surveys, 56(9), 237. https://doi.org/10.1145/3652595

Alshaer

(2025). The role of sensory experiences in evoking emotional responses in virtual reality. Journal of Umm Al-Qura University for Engineering and Architecture, 16(1), 172–184. https://doi.org/10.1007/s43995-025-00099-z

Archer

N. S.

Bluff

Eddy

Nikhil

C. K.

Hazell

Frank

Johnston

(2022). Odour enhances the sense of presence in a virtual reality environment. PLOS ONE, 17(3), e0265039. https://doi.org/10.1371/journal.pone.0265039

Azhin Jutyar

Nahbaz

(2024). Digital humanity as a center for expression in the arts of painting: Kurdistan Region as an example. Qalaai Zanist Scientific Journal, 9(4), 125–150. https://doi.org/10.25212/lfu.qzj.9.4.6

Boucsein

(2012). Methods of electrodermal recording. In Electrodermal activity (pp.87–258). Springer. https://doi.org/10.1007/978-1-4614-1126-0_2

Bradley

M. M.

Lang

P. J.

(1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9

Carhart

Jerger

J. F.

(1959). Preferred method for clinical determination of pure-tone thresholds. Journal of Speech and Hearing Disorders, 24, 330–345. https://doi.org/10.1044/jshd.2404.330

Chen

Y.-S.

Jie

M. T. Y.

Hsieh

C.-E.

Huang

J.-Y.

Han

P.-H.

Hung

Y.-P.

(2024). Reyeal: Implicit gaze-based interaction for creating personal landscape painting in cinematic virtual reality. SIGGRAPH Asia 2024 XR, Article 15. https://doi.org/10.1145/3681759.3688922

D'Andrea

Tischler

Dening

Churchill

(2022). Olfactory stimulation for people with dementia: A rapid review. Dementia (London), 21(5), 1800–1824. https://doi.org/10.1177/14713012221082377

10.

Delong

Noppeney

(2021). Semantic and spatial congruency mould audiovisual integration depending on perceptual awareness. Scientific Reports, 11(1), 10832. https://doi.org/10.1038/s41598-021-90183-w

11.

Đurić

Medić

Ecet

Grgić

Atanacković Jeličić

(2025). Enhancing public space experiences: evaluating perception of digital and digitized sculptures in augmented reality. Applied Sciences, 15(2), 870. https://doi.org/10.3390/app15020870

12.

Ferris

F. L.

3rd Kassoff

Bresnick

G. H.

Bailey

(1982). New visual acuity charts for clinical research. American Journal of Ophthalmology, 94(1), 91–96. https://doi.org/10.1016/0002-9394(82)90197-0

13.

(2024). Research on digital Media installations of emotional interaction based on visual communication design. Journal of Social Science Humanities and Literature, 7(6), 114–141. https://doi.org/10.53469/jsshl.2024.07(06).20

14.

Gehrke

Koselevs

Klug

Gramann

(2025). Neuroadaptive haptics: a proof-of-concept comparing reinforcement learning from explicit ratings and neural signals for adaptive XR systems [Original Research]. Frontiers in Virtual Reality, 6, 1616442. https://doi.org/10.3389/frvir.2025.1616442

15.

Gibbs

J. K.

Gillies

Pan

(2022). A comparison of the effects of haptic and visual feedback on presence in virtual reality. International Journal of Human-Computer Studies, 157, 102717. https://doi.org/10.1016/j.ijhcs.2021.102717

16.

Lewallen

Kinkhabwala

A. A.

Domnisoru

Yoon

Gauthier

J. L.

…Tank

D. W.

(2018). A map-like micro-organization of grid cells in the medial entorhinal Cortex. Cell, 175(3), 736–750.e730. https://doi.org/10.1016/j.cell.2018.08.066

17.

Hartmann

Wirth

Schramm

Klimmt

Vorderer

Gysbers

…Sacau

(2016). The Spatial Presence Experience Scale (SPES): a short self-report measure for diverse media settings. Journal of Media Psychology, 28(1), 1–15. https://doi.org/10.1027/1864-1105/a000137

18.

Hummel

Sekinger

Wolf

S. R.

Pauli

Kobal

(1997). Sniffin’ sticks’: Olfactory performance assessed by the combined testing of odor identification, odor discrimination and olfactory threshold. Chemical Senses, 22(1), 39–52. https://doi.org/10.1093/chemse/22.1.39

19.

Jicol

Cheng

H. Y.

Petrini

O’Neill

(2023). A predictive model for understanding the role of emotion for the formation of presence in virtual reality. PLoS One, 18(3), e0280390. https://doi.org/10.1371/journal.pone.0280390

20.

Lessiter

Freeman

Keogh

Davidoff

(2001). A cross-Media presence questionnaire: The ITC-sense of presence inventory. Presence, 10, 282–297. https://doi.org/10.1162/105474601300343612

21.

(2024). Digital Media intervenes in an immersive exhibition in interactive ScenesMode of perception and interaction in the new era. Applied and Computational Engineering, 99(1), 1–6. https://doi.org/10.54254/2755-2721/99/20251756

22.

Lin

(2025). The Impact of Painting Therapy on Alleviating Anxiety and Enhancing Emotional Expression in Alzheimer's Patients. https://doi.org/10.54254/2753-7064/62/2025.18140

23.

Logeswaran

Munsch

Chong

Y. J.

Ralph

McCrossnan

(2021). The role of extended reality technology in healthcare education: Towards a learner-centred approach. Future Healthcare Journal, 8(1), e79–e84. https://doi.org/10.7861/fhj.2020-0112

24.

Shi

Zhao

Duh

B.-L.

Wang

(2024). PainterAR: A self-painting AR interface for Mobile devices. Computer Animation and Virtual Worlds, 35(6), e2296. https://doi.org/10.1002/cav.2296

25.

Magalhães

Melo

Coelho

A. F.

Bessa

(2024). Affective landscapes: Navigating the emotional impact of multisensory stimuli in virtual reality. IEEE Access, 12, 169955–169976. https://doi.org/10.1109/ACCESS.2024.3499858

26.

Miaomiao

(2024). The planning and design research of the rural architectural landscape of southern Anhui province by virtual reality technology. International Journal of Computational and Experimental Science and Engineering, 10(4), 1511–1523. https://doi.org/10.22399/ijcesen.768

27.

Munyan

B. G.

3rd Neer

S. M.

Beidel

D. C.

Jentsch

(2016). Olfactory stimuli increase presence in virtual environments. PLoS One, 11(6), e0157568. https://doi.org/10.1371/journal.pone.0157568

28.

Nitidara

N. P. A.

Sarwono

Suprijanto

(2024). Exploring cross-modal perception: Audio, visual, and thermal responses to changes in audio-visual stimuli within virtual urban parks. INTER-NOISE and NOISE-CON Congress and Conference Proceedings, 270, 2648–2656. https://doi.org/10.3397/IN_2024_3215

29.

(2025). Integration design of Chinese painting and architectural space based on virtual reality technology. International Journal of High Speed Electronics and Systems, 2540215. https://doi.org/10.1142/S0129156425402153

30.

Snihur

(2024). Аpproaches to creation of digital twins for use in AR/VR galleries. ART-platFORM, 9(1), 428–440. https://doi.org/10.51209/platform.1.9.2024.428-440

31.

Sutcliffe

Gault

(2004). Heuristic evaluation of virtual reality applications. Interacting with Computers, 16(4), 831–849. https://doi.org/10.1016/j.intcom.2004.05.001

32.

Taoum

Bisiaux

Tilquin

Le Guillou

Carrault

(2022). Validity of ultra-short-term HRV analysis using PPG—A preliminary study. Sensors, 22(20), 7995. https://doi.org/10.3390/s22207995

33.

Tewell

Ranasinghe

(2024). A review of olfactory display designs for virtual reality environments. ACM Computing Surveys, 56(11), Article 276. https://doi.org/10.1145/3665243

34.

Uçar

(2024). Exploring the potential of augmented reality (AR) and virtual reality (VR) in developing immersive experiences for education, entertainment, and training: Innovations, applications, and future prospects. Human Computer Interaction, 8, 47. https://doi.org/10.62802/1n99md93

35.

Victorie

D. T. A.

Ajay Kumar

Prakash

(2024). Air canvas: Redefining digital art through gesture based interaction. International Scientific Journal of Engineering and Management, 03(12), 1–7. https://doi.org/10.55041/isjem02175

36.

Warburton

Mon-Williams

Mushtaq

Morehead

J. R.

(2023). Measuring motion-to-photon latency for sensorimotor experiments with virtual reality systems. Behavior Research Methods, 55(7), 3658–3678. https://doi.org/10.3758/s13428-022-01983-5

37.

Xia

Xing

(2024). Digital technology integration innovation and visual symbol design of huizhou ink painting skills under bioengineering. Molecular & Cellular Biomechanics, 21(4), 672. https://doi.org/10.62617/mcb672

38.

Xiao

Wang

Shu

Tan

(2024). Creativity and perception: Unveiling the role of cross-modal audiovisual integration. The Journal of Creative Behavior, 58(3), 460–477. https://doi.org/10.1002/jocb.668

39.

Khanotia

Juni

Sami

Lin

…Rahimpoor-Marnani

(2024). Effectiveness of virtual reality-based well-being interventions for stress reduction in young adults. Systematic Review. JMIR Ment Health, 11, e52186. https://doi.org/10.2196/52186

40.

Yildirim

Globa

Gocer

Brambilla

(2025). Digital smell technologies for the built environment: Evaluating human responses to multisensory stimuli in immersive virtual reality. Building and Environment, 271, 112608. https://doi.org/10.1016/j.buildenv.2025.112608

41.

Yuan

Tang

Zou

Mahmoudinezhad

M. H.

Zhang

Anderson

(2024). Finger painting in VR: Multi-dynamic gestural input for VR painting. SIGGRAPH Asia 2024 XR, Article 6. https://doi.org/10.1145/3681759.3688918

42.

Yujiao

Jie

Ming

(2024). Research on the emotional impact of restorative environments based on facial emotion recognition systems and sora model virtual reality technology. In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), 5638–5645. https://doi.org/10.1109/BIBM62325.2024.10822765

43.

Zhang

Guo

Lee

(2024). Advances in olfactory augmented virtual reality towards future metaverse applications. Nature Communications, 15(1), 6465. https://doi.org/10.1038/s41467-024-50261-9

44.

Zhao

Liu

(2025). Application of human-computer interaction-based virtual reality technology in virtual museums. 121–121. https://doi.org/10.1117/12.3054431

45.

Zhenqi

(2024). Application of home-based XR art experiences for emotional healing in elderly individuals with mobility issues: Eye-tracking and virtual natural landscape Design. Science and Technology of Engineering, Chemistry and Environmental, 2(6), 1–8. https://doi.org/10.61173/0mn00q27

Application of Multi-Sensory Fusion Technology in Digital Landscape Painting: Immersion,Interaction and Emotional Expression

Abstract

Keywords

Get full access to this article

References