Comparing Neural Correlates of Drone Operations in Virtual Reality and 2D Monitor

Abstract

The aim of this study is to investigate the neural correlates during drone control in both VR and 2D monitor conditions. A total of eight participants were instructed to control a drone and perform a heat loss detection task using either VR or a 2D monitor. Electroencephalogram (EEG) signals were recorded and subsequently analyzed using the dynamic causal modeling (DCM) method. By comparing the laterality and the direction of causal connections derived from DCM analysis, the key role of right hemisphere dorsolateral prefrontal cortex (dlPFC) was identified during presence sensing under the VR condition. The results from this preliminary study may be utilized in applications of the augmented cognition program as they provide the basis for monitoring and evaluating the engagement of users.

Keywords

virtual reality sense of presence EEG drone control

Get full access to this article

View all access options for this article.

References

Ashburner

Barnes

Chen

C. C.

Daunizeau

Flandin

Friston

Kiebel

Kilner

Litvak

Moran

(2014). SPM12 manual (Vol. 2464, pp. 475–481). Wellcome Trust Centre for Neuroimaging. https://doi.org/10.1111/j.1365-294X.2006.02813.x

Baumgartner

Speck

Wettstein

Masnari

Beeli

Jäncke

(2008, August 2). Feeling present in arousing virtual reality worlds: Prefrontal brain regions differentially orchestrate presence experience in adults and children. Frontiers in Human Neuroscience, 2, 1–12. https://doi.org/10.3389/neuro.09.008.2008

Bonnefond

Kastner

Jensen

(2017). Communication between brain areas based on nested oscillations. eNeuro, 4(2), ENEURO.0153-16.2017. https://doi.org/10.1523/ENEURO.0153-16.2017

Davis

S. W.

Crowell

C. A.

Beynel

Deng

Lak

Hilbig

S. A.

Lim

Lakhlani

Hilbig

S. A.

Lim

Nguyen

Peterchev

A. V.

Luber

B. M.

Lisanby

S. H.

Appelbaum

L. G.

Cabeza

(2018). Complementary topology of maintenance and manipulation brain networks in working memory. Scientific Reports, 8(1), 17827. https://doi.org/10.1038/s41598-018-35887-2

Deshpande

Kim

(2018). The effects of augmented reality on improving spatial problem solving for object assembly. Advanced Engineering Informatics, 38, 760–775.

de Waele

Baudonnière

P. M.

Lepecq

J. C.

Tran Ba Huy

Vidal

P. P

. (2001). Vestibular projections in the human cortex. Experimental Brain Research, 141(4), 541–551. https://doi.org/10.1007/s00221-001-0894-7

Farzana

Jeffri

Rohaya

Rambli

(2021). A review of augmented reality systems and their effects on mental workload and task performance. Heliyon, 7(3), e06277. https://doi.org/10.1016/j.heliyon.2021.e06277

Forlim

C. G.

Bittner

Mostajeran

Steinicke

Gallinat

Kühn

(2019). Stereoscopic rendering via goggles elicits higher functional connectivity during virtual reality gaming. Frontiers in Human Neuroscience, 13, 365. https://doi.org/10.3389/fnhum.2019.00365

Friedman

N. P.

Robbins

T. W.

(2022). The role of prefrontal cortex in cognitive control and executive function. Neuropsychopharmacology, 47(1), 72–89.

10.

Friston

K. J.

(2011). Functional and effective connectivity: A review. Brain Connectivity, 1(1), 13–36. https://doi.org/10.1089/brain.2011.0008

11.

Hartmann

Wirth

Schramm

Klimmt

Vorderer

Gysbers

Böcking

Ravaja

Laarni

Saari

Gouveia

Maria Sacau

(2016). The spatial presence experience scale (SPES): A short self-report measure for diverse media settings. Journal of Media Psychology, 28, 1–15. https://doi.org/10.1027/1864-1105/a000137

12.

Huang

Pugh

Z. H.

Kim

Nam

C. S.

(2024, March). Brain dynamics of mental workload in a multitasking context: Evidence from dynamic causal modeling. Computers in Human Behavior, 152, 108043. https://doi.org/10.1016/j.chb.2023.108043

13.

Jäncke

Cheetham

Baumgartner

(2009). Virtual reality and the role of the prefrontal cortex in adults and children. Frontiers in Neuroscience, 3, 52–59. https://doi.org/10.3389/neuro.01.006.2009

14.

Kiebel

David

Friston

K. J.

Kiebel

S. J.

David

Friston

K. J.

(2006). Dynamic causal modelling of evoked responses in EEG/MEG with lead field parameterization. NeuroImage, 30(4), 1273–1284. https://doi.org/10.1016/j.neuroimage.2005.12.055

15.

Kober

S. E.

Kurzmann

Neuper

(2012). Cortical correlate of spatial presence in 2D and 3D interactive virtual reality: An EEG study. International Journal of Psychophysiology, 83(3), 365–374. https://doi.org/10.1016/j.ijpsycho.2011.12.003

16.

Kulke

Pasqualette

(2024). Learning from 360-degree videos on a VR headset compared to 2D videos on a monitor. Cogent Education, 11(1). https://doi.org/10.1080/2331186x.2024.2355386

17.

Makransky

Petersen

G. B.

(2021). The cognitive affective model of immersive learning (camil): A theoretical research-based model of learning in immersive virtual reality. Educational Psychology Review, 33(3), 937–958.

18.

Makransky

Terkildsen

T. S.

Mayer

R. E.

(2019). Adding immersive virtual reality to a science lab simulation causes more presence but less learning. Learning and Instruction, 60, 225–236. https://doi.org/10.1016/j.learninstruc.2017.12.007

19.

Mueller

Luehrs

Baecke

Adolf

Luetzkendorf

Luchtmann

Bernarding

(2012). Building virtual reality fMRI paradigms: A framework for presenting immersive virtual environments. Journal of Neuroscience Methods, 209(2), 290–298. https://doi.org/10.1016/j.jneumeth.2012.06.025

20.

Penny

W. D.

Friston

K. J.

Ashburner

J. T.

Kiebel

S. J.

Nichols

T. E.

(2011). Statistical parametric mapping: The analysis of functional brain images. Elsevier.

21.

Penny

W. D.

Stephan

K. E.

Daunizeau

Rosa

M. J.

Friston

K. J.

Schofield

T. M.

Leff

A. P.

(2010). Comparing families of dynamic causal models. PLoS Computational Biology, 6(3), e1000709.

22.

Pfeiffer

Scaramuzza

(2021). Human-piloted drone racing: Visual processing and control. IEEE Robotics and Automation Letters, 6, 3467–3474.

23.

Sharbrough

Chatrian

Lesser

Luders

Nuwer

Picton

(1991). American electroencephalographic society guidelines for Standard Electrode Position Nomenclature. Journal of Clinical Neurophysiology, 8(2), 200–202.

24.

Stanney

K. M.

Schmorrow

D. D.

Johnston

Fuchs

Jones

Hale

K. S.

Ahmad

Young

(2009). Augmented cognition: An overview. Reviews of Human Factors and Ergonomics, 5, 195–224. https://doi.org/10.1518/155723409x448062

25.

Stephan

K. E.

Friston

K. J.

(2009). Functional connectivity. In Squire

L. R.

(Ed.), Encyclopedia of Neuroscience (pp. 391–397). Academic Press.

26.

Stephan

K. E.

Friston

K. J.

(2010). Analyzing effective connectivity with functional magnetic resonance imaging. https://doi.org/10.1002/wcs.58

27.

Stoffregen

T. A.

Bardy

B. G.

Smart

Pagulayan

(2003). On the nature and evaluation of fidelity in virtual environments. In Hettinger

L. J.

Haas

(Eds.) Virtual and adaptive environments: Applications, implications, and human performance issues (pp. 21–46). Lawrence Erlbaum Associates. https://doi.org/10.1201/9781410608888.pt1

28.

Wig

G. S.

Schlaggar

B. L.

Petersen

S. E.

(2011). Concepts and principles in the analysis of brain networks. Annals of the New York Academy of Sciences, 1224(1), 126–146. https://doi.org/10.1111/j.1749-6632.2010.05947.x

29.

(2020). Effectiveness of immersive virtual reality using head-mounted displays on learning performance: A meta-analysis. British Journal of Educational Technology, 51(6), 1991–2005. https://doi.org/10.1111/bjet.13023

30.

Chen

Gama

Riar

Hamari

(2023). The challenges of entering the metaverse: An experiment on the effect of extended reality on workload. Information Systems Frontiers, 25(2), 659–680. https://doi.org/10.1007/s10796-022-10244-x