Examining User Interactions With Signaling Elements in a Virtual Reality Learning Application

Abstract

This preliminary study examined how users leveraged three different types of signaling elements in Scale Worlds, an immersive virtual reality (IVR) application designed to improve size and scale cognition. Signaling elements, which are instructional cues in the form of graphics, colors, sounds, or text in IVR, may improve learning outcomes by enhancing related cognitive processes. However, it is unclear the extent to which learners utilize these signaling elements in practice. A think-aloud protocol was used to examine how participants engaged with signaling elements, with thematic analysis suggesting that numerical measures were a particularly salient cue for conceptualizing the size and scale of entities in IVR. These findings can guide design decisions for future work on educational IVR in the context of size and scale cognition or STEM education, as implementing numerical measures to facilitate mathematical reasoning in IVR environments may bolster learning outcomes related to numeracy and conceptual understanding.

Keywords

virtual reality virtual elements signaling elements scale learning

Get full access to this article

View all access options for this article.

References

Barsalou

L. W.

(1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22(4), 577–660. https://doi.org/10.1017/S0140525X99002149

Braun

Clarke

(2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. https://doi.org/10.1191/1478088706qp063oa

Cheng

Harper-Gampp

Planchart

Dunning

Peterson

Delgado

Chen

K. B.

(2024). Study of graphic armatures, multimodal cues, and numeric measures in virtual reality on learners’ performance and workload. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 68(1), 1229–1234. https://doi.org/10.1177/10711813241269236

Delgado

Peterson

(2018). 1692 an enhanced framework for scale cognition leveraging visual metaphor theory and analogical reasoning theory. In Kay

Luckin

(Eds.), Rethinking learning in the digital age: Making the learning sciences count, 13th international conference of the learning sciences (ICLS) 2018, volume 3. International Society of the Learning Sciences. https://doi.org/10.22318/cscl2018.1607

Ericsson

K. A.

Crutcher

R. J.

(1991). Introspection and verbal reports on cognitive processes—two approaches to the study of thinking: A response to Howe. New Ideas in Psychology, 9(1), 57–71. https://doi.org/10.1016/0732-118X(91)90041-J

Gibson

J. J.

(2014). The ecological approach to visual perception: Classic edition. Psychology Press.

Harper-Gampp

Delgado

Peterson

Chen

K. B.

(2023a, April). Designing and developing an instrument to assess scale cognition [Paper presentation]. National association for research in science teaching annual conference, Chicago, IL, USA.

Harper-Gampp

Delgado

Peterson

Chen

K. B.

(2023b, April). Refining a panel of experts validation methodology for instrument development [Roundtable presentation]. American education research association, Chicago, IL, USA.

Hart

S. G.

Staveland

L. E.

(1988). Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Hancock

P. A.

Meshkati

(Eds.), Advances in psychology (Vol. 52, pp. 139–183). North-Holland.

10.

Feng

Zhu

Wang

(2023). Effect of summarizing scaffolding and textual cues on learning performance, mental model, and cognitive load in a virtual reality environment: An experimental study. Computers & Education, 200, Article 104793. https://doi.org/10.1016/j.compedu.2023.104793

11.

Liu

Yang

Huang

(2022). Impacts of cues on learning and attention in immersive 360-degree video: An eye-tracking study. Frontiers in Psychology, 12, Article 792069. https://doi.org/10.3389/fpsyg.2021.792069

12.

Magana

A. J.

Brophy

S. P.

Bryan

L. A.

(2012). An integrated knowledge framework to characterize and scaffold size and scale cognition (FS2C). International Journal of Science Education, 34(14), 2181–2203. https://doi.org./10.1080/09500693.2012.715316

13.

Mautone

P. D.

Mayer

R. E.

(2001). Signaling as a cognitive guide in multimedia learning. Journal of Educational Psychology, 93(2), 377–389. https://doi.org/10.1037/0022-0663.93.2.377

14.

Peterson

Delgado

Tang

K. S.

Bordas

Norville

(2021). A taxonomy of cognitive image functions for science curriculum materials: Identifying and creating ‘performative’ visual displays. International Journal of Science Education, 43(2), 314–343. https://doi.org/10.1080/09500693.2020.1868609

15.

Planchart

Dunning

Peterson

Delgado

Chen

K. B.

(2024). Isolating and addressing theoretically-grounded limitations from the rapid translation of interaction design across media platforms. In Gray

Ciliotta Chehade

Hekkert

Forlano

Ciuccarelli

Lloyd

(Eds.), Proceedings of DRS2024: Boston (pp. 23–28). Design Research Society. https://doi.org/10.21606/drs.2024.1187

16.

Rodemer

Lindner

M. A.

Eckhard

Graulich

Bernholt

(2022). Dynamic signals in instructional videos support students to navigate through complex representations: An eye-tracking study. Applied Cognitive Psychology, 36(4), 852–863. https://doi.org/10.1002/acp.3973

17.

Schneider

Beege

Nebel

Rey

G. D.

(2018). A meta-analysis of how signaling affects learning with media. Educational Research Review, 23, 1–24. https://doi.org/10.1016/j.edurev.2017.11.001

18.

Tannert

Eitel

Marder

Seidel

Renkl

Glogger-Frey

(2023). How can signaling in authentic classroom videos support reasoning on how to induce learning strategies? Frontiers in Education, 8, Article 974696. https://doi.org/10.3389/feduc.2023.974696

19.

Wagemans

Elder

J. H.

Kubovy

Palmer

S. E.

Peterson

M. A.

Singh

von der Heydt

(2012). A century of Gestalt psychology in visual perception: I. Perceptual grouping and figure–ground organization. Psychological Bulletin, 138(6), 1172–1217. https://doi.org/10.1037/a0029333

20.

Won

Ungu

D. A. K.

Matovu

Treagust

D. F.

Tsai

C.-C.

Park

Mocerino

Tasker

(2023). Diverse approaches to learning with immersive Virtual Reality identified from a systematic review. Computers & Education, 195, Article 104701. https://doi.org/10.1016/j.compedu.2022.104701

21.

(2020). Effectiveness of immersive virtual reality using head-mounted displays on learning performance: A meta-analysis. British Journal of Educational Technology, 51(6), 1991–2005. https://doi.org/10.1111/bjet.13023

22.

Sekelsky

Peterson

Gampp

Delgado

Chen

K. B.

(2022). Immersive virtual environment for scale cognition and learning: Expert-based evaluation for balancing usability versus cognitive theories. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 66(1), 1972–1976. https://doi.org/10.1177/1071181322661094

23.

Liu

Johnson-Glenberg

M. C.

Han

(2023). Promoting musical instrument learning in virtual reality environment: Effects of embodiment and visual cues. Computers & Education, 198, Article 104764. https://doi.org/10.1016/j.compedu.2023.104764