While virtual reality (VR) has great educational potential, current implementations predominantly emphasize audiovisual simulations, neglecting olfactory and haptic stimuli and thereby compromising the physical fidelity of VR learning environments. Meanwhile, conflicting findings on the impact of immersion level on learning outcomes, coupled with limited research exploring the interaction between physical fidelity and immersion level, have impeded the widespread adoption of VR in education. Therefore, this study aimed to examine the effects of physical fidelity (multisensory stimuli vs. no multisensory stimuli) and immersion level (immersive VR vs. desktop VR) on learning experiences (cognitive load, learning motivation, and quality of experience) and outcomes (procedural, factual, and conceptual knowledge) in VR-based fire-safety education. A two-by-two factorial experiment was conducted with 120 Chinese university students. The results showed that multisensory stimuli significantly reduced the intrinsic cognitive load of learners. Immersive VR surpassed desktop VR for bolstering learning motivation, particularly in terms of attention, relevance, and satisfaction. Physical fidelity and immersion level have significant interaction effects on long-term retention of conceptual knowledge and satisfaction. Incorporating multisensory stimuli into immersive VR enhances the quality of experience. Our findings have significant implications and value for the application of multisensory stimuli and VR technology in education.
AdemoyeO. A.GhineaG. (2013). Information recall task impact in olfaction-enhanced multimedia. ACM Transactions on Multimedia Computing, Communications, and Applications, 9(3), 1–16. https://doi.org/10.1145/2487268.2487270
2.
AndersonL. W.KrathwohlD.AirasianP.CruikshankK. A.MayerR. E.PintrichP.RathsJ.WittrockM. C. (2001). A taxonomy for learning, teaching and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.
3.
BiT.LyonsR.FoxG.MunteanG. (2021). Improving student learning satisfaction by using an innovative dash-based multiple sensorial media delivery solution. IEEE Transactions on Multimedia, 23, 3494–3505. https://doi.org/10.1109/tmm.2020.3025669
4.
ButtussiF.ChittaroL. (2018). Effects of different types of virtual reality display on presence and learning in a safety training scenario. IEEE Transactions on Visualization and Computer Graphics, 24(2), 1063–1076. https://doi.org/10.1109/tvcg.2017.2653117
5.
ChaM.HanS.LeeJ.ChoiB. (2012). A virtual reality based fire training simulator integrated with fire dynamics data. Fire Safety Journal, 50, 12–24. https://doi.org/10.1016/j.firesaf.2012.01.004
6.
ChampneyR. K.StanneyK. M.MilhamL.CarrollM. B.CohnJ. V. (2017). An examination of virtual environment training fidelity on training effectiveness. International Journal of Learning Technology, 12(1), 42–65. https://doi.org/10.1504/IJLT.2017.083997
7.
ChandlerP.SwellerJ. (1991). Cognitive load theory and the format of instruction. Cognition and Instruction, 8(4), 293–332. https://doi.org/10.1207/s1532690xci0804_2
8.
CohenJ. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Routledge.
9.
CovaciA.GhineaG.LinC.HuangS.ShihJ. (2018a). Multisensory games-based learning-lessons learnt from olfactory enhancement of a digital board game. Multimedia Tools and Applications, 77(16), 21245–21263. https://doi.org/10.1007/s11042-017-5459-2
10.
CovaciA.TrestianR.SalemeE. B.ComsaI.AssresG.SantosC. A. S.GhineaG. (2019). 360° mulsemedia: A way to improve subjective QoE in 360° videos. In Proceedings of the 27th ACM international conference on multimedia (pp. 2378–2386). Association for Computing Machinery (ACM). https://doi.org/10.1145/3343031.3350954
11.
CovaciA.ZouL.TalI.MunteanG.GhineaG. (2018b). Is multimedia multisensorial? - A review of mulsemedia systems. ACM Computing Surveys, 51(5), 1–35. https://doi.org/10.1145/3233774
12.
CummingsJ. J.BailensonJ. N. (2016). How immersive is enough? A meta-analysis of the effect of immersive technology on user presence. Media Psychology, 19(2), 272–309. https://doi.org/10.1080/15213269.2015.1015740
13.
EdwardsB. I.BielawskiK. S.PradaR.CheokA. D. (2019). Haptic virtual reality and immersive learning for enhanced organic chemistry instruction. Virtual Reality, 23(4), 363–373. https://doi.org/10.1007/s10055-018-0345-4
14.
Garcia-RuizM. A.KapralosB.Rebolledo-MendezG. (2021). An overview of olfactory displays in education and training. Multimodal Technologies and Interaction, 5(10), 64. https://doi.org/10.3390/mti5100064
15.
GhineaG.AdemoyeO. A. (2010). Perceived synchronization of olfactory multimedia. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans, 40(4), 657–663. https://doi.org/10.1109/tsmca.2010.2041224
16.
GhineaG.TimmererC.LinW.GulliverS. R. (2014). Mulsemedia: State of the art, perspectives, and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications, 11(1), 1–23. https://doi.org/10.1145/2617994
17.
HarpS. F.MayerR. E. (1998). How seductive details do their damage: A theory of cognitive interest in science learning. Journal of Educational Psychology, 90(3), 414–434. https://doi.org/10.1037/0022-0663.90.3.414
18.
HoC.LinT.ChangC. (2023). Interactive multi-sensory and volumetric content integration for music education applications. Multimedia Tools and Applications, 82(4), 4847–4862. https://doi.org/10.1007/s11042-022-12314-3
19.
JensenL.KonradsenF. (2018). A review of the use of virtual reality head-mounted displays in education and training. Education and Information Technologies, 23(4), 1515–1529. https://doi.org/10.1007/s10639-017-9676-0
20.
Kani-ZabihiE.HussainN.MesfinG.CovaciA.GhineaG. (2021). On the influence of individual differences in cross-modal mulsemedia QoE. Multimedia Tools and Applications, 80(2), 2377–2394. https://doi.org/10.1007/s11042-020-09757-x
21.
KimK.RosenthalM. Z.ZielinskiD. J.BradyR. (2014). Effects of virtual environment platforms on emotional responses. Computer Methods and Programs in Biomedicine, 113(3), 882–893. https://doi.org/10.1016/j.cmpb.2013.12.024
22.
KlepschM.SchmitzF.SeufertT. (2017). Development and validation of two instruments measuring intrinsic, extraneous, and germane cognitive load. Frontiers in Psychology, 8, 1997. https://doi.org/10.3389/fpsyg.2017.01997
23.
KlingenbergS.JørgensenM. L. M.DandanellG.SkriverK.MottelsonA.MakranskyG. (2020). Investigating the effect of teaching as a generative learning strategy when learning through desktop and immersive VR: A media and methods experiment. British Journal of Educational Technology, 51(6), 2115–2138. https://doi.org/10.1111/bjet.13029
24.
KwonC. (2021). A study on the verification of the effect of sensory extension through cutaneous sensation on experiential learning using VR. Virtual Reality, 25(1), 19–30. https://doi.org/10.1007/s10055-020-00435-9
25.
LawsonG.RoperT.ShawE.HsiehM.CobbS. (2020). Multimodal virtual environments: An opportunity to improve fire safety training?Policy and Practice in Health and Safety, 18(2), 155–168. https://doi.org/10.1080/14773996.2020.1796085
26.
LeeE. A.WongK. W. (2014). Learning with desktop virtual reality: Low spatial ability learners are more positively affected. Computers & Education, 79, 49–58. https://doi.org/10.1016/j.compedu.2014.07.010
27.
LeeE. A.WongK. W.FungC. C. (2010). How does desktop virtual reality enhance learning outcomes? A structural equation modeling approach. Computers & Education, 55(4), 1424–1442. https://doi.org/10.1016/j.compedu.2010.06.006
28.
LiW.QianL.FengQ.LuoH. (2024). Using olfactory cues in text materials benefits delayed retention and schemata construction. Scientific Reports, 14(1), Article 17819. https://doi.org/10.1038/s41598-024-68885-8
29.
LiW.RenX.QianL.LuoH.LiuB. (2023). Uncovering the effect of classroom climates on learning experience and performance in a virtual environment. Interactive Learning Environments, 32(8), 4256–4269. https://doi.org/10.1080/10494820.2023.2195450
30.
LoorbachN.PetersO.KarremanJ.SteehouderM. (2015). Validation of the Instructional Materials Motivation Survey (IMMS) in a self-directed instructional setting aimed at working with technology. British Journal of Educational Technology, 46(1), 204–218. https://doi.org/10.1111/bjet.12138
31.
LovreglioR.DuanX.RahoutiA.PhippsR.NilssonD. (2021). Comparing the effectiveness of fire extinguisher virtual reality and video training. Virtual Reality, 25(1), 133–145. https://doi.org/10.1007/s10055-020-00447-5
32.
LuiM.McEwenR.MullallyM. (2020). Immersive virtual reality for supporting complex scientific knowledge: Augmenting our understanding with physiological monitoring. British Journal of Educational Technology, 51(6), 2181–2199. https://doi.org/10.1111/bjet.13022
33.
LuoH.LiG.FengQ.YangY.ZuoM. (2021). Virtual reality in K‐12 and higher education: A systematic review of the literature from 2000 to 2019. Journal of Computer Assisted Learning, 37(3), 887–901. https://doi.org/10.1111/jcal.12538
34.
LwinM. O.MorrinM.KrishnaA. (2010). Exploring the superadditive effects of scent and pictures on verbal recall: An extension of dual coding theory. Journal of Consumer Psychology, 20(3), 317–326. https://doi.org/10.1016/j.jcps.2010.04.001
35.
MaasM. J.HughesJ. M. (2020). Virtual, augmented and mixed reality in K-12 education: A review of the literature. Technology, Pedagogy and Education, 29(2), 231–249. https://doi.org/10.1080/1475939x.2020.1737210
36.
MaganaA. J.SerranoM. I.RebelloN. S. (2019). A sequenced multimodal learning approach to support students’ development of conceptual learning. Journal of Computer Assisted Learning, 35(4), 516–528. https://doi.org/10.1111/jcal.12356
37.
MakranskyG.LilleholtL. (2018). A structural equation modeling investigation of the emotional value of immersive virtual reality in education. Educational Technology Research & Development, 66(5), 1141–1164. https://doi.org/10.1007/s11423-018-9581-2
38.
MakranskyG.PetersenG. B. (2019). Investigating the process of learning with desktop virtual reality: A structural equation modeling approach. Computers & Education, 134, 15–30. https://doi.org/10.1016/j.compedu.2019.02.002
39.
MakranskyG.PetersenG. B. (2021). The Cognitive Affective Model Of Immersive Learning (CAMIL): A theoretical research-based model of learning in immersive virtual reality. Educational Psychology Review, 33(3), 937–958. https://doi.org/10.1007/s10648-020-09586-2
40.
MakranskyG.TerkildsenT. S.MayerR. E. (2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learning and Instruction, 60, 225–236. https://doi.org/10.1016/j.learninstruc.2017.12.007
41.
MayerR.MayerR. E. (2005). The Cambridge handbook of multimedia learning. Cambridge University Press.
42.
McMahanA. (2003). Immersion, engagement, and presence: A method for analyzing 3-D video games. In WolfM. J. P.BernardP. (Eds.), The video game theory reader (pp. 67–86). Routledge.
43.
MesfinG.HussainN.Kani‐ZabihiE.CovaciA.SalemeE. B.GhineaG. (2020). QoE of cross-modally mapped mulsemedia: An assessment using eye gaze and heart rate. Multimedia Tools and Applications, 79(11-12), 7987–8009. https://doi.org/10.1007/s11042-019-08473-5
44.
MillerH. L.BugnariuN. L. (2016). Level of immersion in virtual environments impacts the ability to assess and teach social skills in autism spectrum disorder. Cyberpsychology, Behavior, and Social Networking, 19(4), 246–256. https://doi.org/10.1089/cyber.2014.0682
45.
MohanaM.ValliammalN.SuvethaV.KrishnaveniM.SubashiniP.GhineaG. (2023). A study on technology-enhanced mulsemedia learning for enhancing learner’s experience in e-learning. In 2023 international conference on network, multimedia and information technology (NMITCON) (pp. 01–06). IEEE. https://doi.org/10.1109/NMITCON58196.2023.10275964
46.
MorélotS.GarrigouA.DedieuJ.N’KaouaB. (2021). Virtual reality for fire safety training: Influence of immersion and sense of presence on conceptual and procedural acquisition. Computers & Education, 166, Article 104145. https://doi.org/10.1016/j.compedu.2021.104145
NarcisoD.BessaM.MeloM.Vasconcelos-RaposoJ. (2019). Virtual reality for training-the impact of smell on presence, cybersickness, fatigue, stress and knowledge transfer. In 2019 international conference on graphics and interaction (ICGI) (pp. 115–121). IEEE. https://doi.org/10.1109/ICGI47575.2019.8955071
49.
NarcisoD.MeloM.RodriguesS.CunhaJ. P.Vasconcelos-RaposoJ.BessaM. (2023). Studying the influence of multisensory stimuli on a firefighting training virtual environment. IEEE Transactions on Visualization and Computer Graphics, 30(7), 4122–4136. https://doi.org/10.1109/TVCG.2023.3251188
50.
NovakM.PhelanS.LewalterD.SchwanS. (2020). There is more to touch than meets the eye: Haptic exploration in a science museum. International Journal of Science Education, 42(18), 3026–3048. https://doi.org/10.1080/09500693.2020.1849855
ParongJ.MayerR. E. (2021). Cognitive and affective processes for learning science in immersive virtual reality. Journal of Computer Assisted Learning, 37(1), 226–241. https://doi.org/10.1111/jcal.12482
53.
RaheelA.MajidM.AnwarS. M. (2021). Dear-Mulsemedia: Dataset for emotion analysis and recognition in response to multiple sensorial media. Information Fusion, 65, 37–49. https://doi.org/10.1016/j.inffus.2020.08.007
54.
RauP. P.ZhengJ.WeiY. (2020). Distractive effect of multimodal information in multisensory learning. Computers & Education, 144, Article 103699. https://doi.org/10.1016/j.compedu.2019.103699
RuppM. A.OdetteK.KozachukJ.MichaelisJ. R.SmitherJ. A.McConnellD. S. (2019). Investigating learning outcomes and subjective experiences in 360-degree videos. Computers & Education, 128, 256–268. https://doi.org/10.1016/j.compedu.2018.09.015
57.
SalemeE. B.CovaciA.AssresG.ComşaI.TrestianR.SantosC. A. S.GhineaG. (2021). The influence of human factors on 360° mulsemedia QoE. International Journal of Human-Computer Studies, 146, Article 102550. https://doi.org/10.1016/j.ijhcs.2020.102550
ShawE.RoperT.NilssonT.LawsonG.CobbS. V.MillerD. (2019). The heat is on: Exploring user behaviour in a multisensory virtual environment for fire evacuation. In Proceedings of the 2019 CHI conference on human factors in computing systems (pp. 1–13). Association for Computing Machinery (ACM). https://doi.org/10.1145/3290605.3300856
60.
SlaterM.WilburS. (1997). A framework for immersive virtual environments (FIVE): Speculations on the role of presence in virtual environments. Presence: Teleoperators and Virtual Environments, 6(6), 603–616. https://doi.org/10.1162/pres.1997.6.6.603
61.
SmithS.EricsonE. (2009). Using immersive game-based virtual reality to teach fire-safety skills to children. Virtual Reality, 13(2), 87–99. https://doi.org/10.1007/s10055-009-0113-6
62.
SouthgateE.SmithS. P.CividinoC.SaxbyS.KilhamJ.EatherG.ScevakJ.SummervilleD. A.BuchananR.BerginC. (2019). Embedding immersive virtual reality in classrooms: Ethical, organisational and educational lessons in bridging research and practice. International Journal of Child-Computer Interaction, 19, 19–29. https://doi.org/10.1016/j.ijcci.2018.10.002
63.
TalI.ZouL.FarrenM.MunteanG. (2020). Mulsemedia in education: A case study on learner experience, motivation and knowledge gain. In Proceedings of the 12th international conference on computer supported education (pp. 180–187). SciTePress. https://doi.org/10.5220/0009579701800187
64.
TaranillaR. V.Cózar-GutiérrezR.González‐CaleroJ. A.CirugedaI. L. (2022). Strolling through a city of the Roman Empire: An analysis of the potential of virtual reality to teach history in primary education. Interactive Learning Environments, 30(4), 608–618. https://doi.org/10.1080/10494820.2019.1674886
65.
WareingJ.LawsonG.AbdullahC.RoperT. (2018). User perception of heat source location for a multisensory fire training simulation. In 2018 10th computer science and electronic engineering (CEEC) (pp. 214–218). IEEE. https://doi.org/10.1109/CEEC.2018.8674211
66.
WebbM.TraceyM.HarwinW.TokatliO.HwangF.JohnsonR.BarrettN.JonesC. (2022). Haptic-enabled collaborative learning in virtual reality for schools. Education and Information Technologies, 27(1), 937–960. https://doi.org/10.1007/s10639-021-10639-4
67.
XieB.LiuH.AlghofailiR.ZhangY.JiangY.LoboF. D.LiC.LiW.HuangH.AkdereM.MousasC.YuL. (2021). A review on virtual reality skill training applications. Frontiers in Virtual Reality, 2, Article 645153. https://doi.org/10.3389/frvir.2021.645153
68.
YuanZ.GhineaG.MunteanG. (2015). Beyond multimedia adaptation: Quality of experience-aware multi-sensorial media delivery. IEEE Transactions on Multimedia, 17(1), 104–117. https://doi.org/10.1109/tmm.2014.2371240
69.
ZhangW.MaW.BrandaoM.KaberD. B.BloomfieldP.SwangnetrM. (2016). Biometric validation of a virtual reality-based psychomotor test for motor skill training. Assistive Technology, 28(4), 233–241. https://doi.org/10.1080/10400435.2016.1165758
