Learning Scale in Virtual Reality: Experiences and Perception of Immersive Technology at a Public Middle School

Abstract

Students have misconceptions of size and scale cognition as they confuse molecules and cells. The team deployed a virtual reality (VR) learning tool, namely Scale Worlds, for K-12 students to use at school. The present work aimed to examine the experience and perception of immersive technology, and logistical challenges of integrating Scale Worlds into a science class. Ten students and three teachers were included in this study with informed consent. Scale Worlds was introduced as part of a science class and then semi-structured interview was conducted. Students’ experience with VR technology included physiological discomfort, psychological nervousness and uncertainty of technology, personal abilities and unfamiliarity, and hardware latency. However, students perceived Scale Worlds to be a useful tool that helped them visualize entities of different sizes, and allowed them to work collaboratively. Teachers expressed desire for more exciting content in Scale Worlds and further usability enhancement, as well as need for additional material preparation time. The team planned to return to the same middle school for another round of in-field study after addressing some of the reported challenges.

Introduction

The application of virtual reality (VR) technology is constantly being explored in different domains to attain positive impacts and outcomes for individuals and the society. It includes learning and education, healthcare and rehabilitation, exercise and training, and workplace and employment training (Gampp et al., 2022; Jensen & Konradsen, 2018; Matamala-Gomez et al., 2022; Osti et al., 2021; Rose et al., 2018). The present work is in the context of education, specifically in numeracy and size and scale cognition. Research has shown that K-12 students have inaccurate conceptualization of size and scale (Tretter, Jones, & Minogue, 2006; Tretter, Jones, Andre, et al., 2006). For instance, students confuse molecules and cells (Arnold, 1983). The misconception can have profound effects in appreciating advanced STEM concepts, such as astrophysics, biology, and chemistry, or other subjects that involve entities of more extreme sizes. While the overarching goal of the research team is to support size and scale cognition of learners, the present study was scoped to be understanding the experience and perception of VR technology during initial deployment in classrooms.

To date, the research team has conducted two usability evaluations on Scale Worlds, including a heuristic evaluation with human factors experts (Wu et al., 2022) and a formative evaluation with college students (Wu et al., 2023) to ensure the usability Scale Worlds prior to deployment in the classroom. Despite the lab-based evaluations for enhancing the overall usability, the integration of Scale Worlds experiencing through head-mounted displays at a public middle school could encounter different human factors and logistical challenges. It was also unclear how students perceive this type of educational experience, and how receptive the teachers were toward this mode of instructional delivery.

Researchers have studied students and/or teachers perception toward VR technologies and the new instructional approach in K-12 classrooms. While these were first steps to in-classroom research, many of which utilized existing off-the-shelf VR content (Bennett & Saunders, 2019; Yıldırım et al., 2020), which was convenient but might been developed with pedagogical rationales considered. Furthermore, the human factors concerning the interactions between the performance environment (e.g., classroom) and the variability of users should be further understood.

The present work holds significance in the application of an immersive learning environment to support the understanding of size and scale of a cohort (e.g., middle school students) in a formal educational setting. Furthermore, the deployed immersive learning environment, namely Scale Worlds, was not only built by the team (as opposed to those purchased from app stores) but also emphasizes content that was grounded embodied cognition theory and recognizing scale as a cross-cutting concept by the Next Generation Science Standards (National Research Council, 2012).

The present work aimed to understand students and teachers experience and perception regarding the deployment and utility of Scale Worlds in classroom setting with middle school students. Specifically, eighth graders and their sciences teachers were involved in this study because using exponents to express extreme sizes (10^×) is fundamental in mathematics (National Governors Association Center for Best Practices, Council of Chief State School Officers, 2010). Students first experienced Scale Worlds as part of their science class, and then both students and teachers were interviewed about their experiences and perception of Scale Worlds and VR technology for learning. Findings of this work may help reveal the long-term usage feasibility and practicality VR in K-12 public schools.

Methods

Participants

Ten eighth grade students and three teachers were included in this initial study. Table 1 lists general student demographics. All spoke English. The distribution of students demographics in this study had higher proportion of Black or African American and more students of Hispanic origin than the county (U.S. Census Bureau Quick Facts: Wake County, North Carolina, n.d.). All three teachers taught eighth grade science (One female and two males; one White, one Black or African American, one prefer not to answer; one of Hispanic origin). Informed consent was obtained from students’ parents and middle school teachers, and informed assent was obtained from students. The study protocol was approved by the North Carolina State University Institutional Review Board.

Table 1.

Student Gender, Race, and Ethnicity. The Three Selected “Prefer Not to Answer” for Race Have All Selected Hispanic Origin.

Gender	Race	Ethnicity
Seven female	Five Black or African American	Four Hispanic, Latino, or Spanish origin.
Three male	One White	Six not of Hispanic, Latino, or Spanish origin.
	One, two, or more races
	Three prefer not to answer

Equipment

Eight VR head-mounted displays, or HMDs (Quest 2, Meta, Menlo Park, CA), were brought to a Title I (i.e., high percentages low-income families) public middle school. Scale Worlds was developed using Unity 3D (ver 2021, Unity Technologies). A tablet computer was available (not specific brand but anything similar to a Chromebook). A voice recorder was used to record the semi-structured interviews. While not within the scope of the present work and analysis, but for transparency, the students underwent a baseline pre-test and a post-test regarding their understanding of size and scale using a validate tool called Assessment of Size and Scale Cognition, or ASSC (Gampp et al., 2023).

Procedure

Upon inform assents, all students completed the ASSC baseline pre-test and then worked in teams of three to four students to experience Scale Worlds, which was introduced to the students as part of their science class that discussed natural gas. Specific connections were made between Scale Worlds and natural, including the structure and size of a methane (small natural gas molecule) and the extension, long infrastructure that carried it (long conduits) through the U.S. The session concluded with ASSC post-test and a semi-structured interview. The teachers were the instructional support and were also interviewed at the end of the session.

Within Scale Worlds, students first completed an orientation tutorial along a white bridge (Figure 1) where they became acquainted with the VR controllers and possible button clicks and interactions. Participants then entered the Human World (Figure 2). They had ability to teleport to different locations and visualize entities of different sizes (Figure 3) from different angles. They could also “grow” or “shrink” to a different scale world that was facilitated by a series of animations to suggest the visual illusion of growing or shrinking. Students within each group took turns to try Scale Worlds while others watched it through casting on the tablet computer.

Figure 1.

Screenshot from scale worlds (SW) showing the “orientation bridge” on which a learner would receive a tutorial of the learning environment and interaction techniques, such as clicking, aiming, and teleporting.

Figure 2.

Screenshot of the first “world” in which a learner entered once the tutorial was completed. This is the human world, which shows an astronaut (human) as the primary scientific entity. A learner can see a right whale afar, which is a scientific entity that is 10× larger (longer) than the human. A learner can also see an American robin, which is a scientific entity that is 10× smaller (shorter) than the human.

Figure 3.

Screenshot showing the American Robin World, which is one world below the human world. If a human was assumed to be 1×100 m, a robin was 1×10^-1 m. In the American Robin World, a learner can see the legs of an astronaut afar, which is a scientific entity that is 10× larger (longer) than the robin. A learner can see an acorn, which is the scientific entity of the White Oak Acorn World that is 10× smaller (shorter) than the robin.

Variables and Analysis

The data of interest were responses from the semi-structured interviews with students and teachers. The responses were coded and similar responses gave rise to themes. Semi-structured interview questions for students included: (a) Tell me about your experience completing the individual activity in Scale Worlds using the head-mounted display. (b) How would you describe your experience in Scale Worlds during the whole-class activity? (c) Would you recommend that your teachers use virtual reality to help support your learning?

Semi-structured interview questions for teachers were: (a) Tell me about your experience with virtual reality in the classroom. (b) Evaluate the instructional lesson using Scale Worlds. Would you consider using it in the future? Would you recommend it to other teachers? (c) How was the class using Scale Worlds different or similar to a typical lesson? (d) Based on implementation, would you consider any changes to Scale Worlds or the lesson? (e) Talk to me about the students’ experience using Scale Worlds. Do you think it helped them foster connections?

Results

Students Experiences

There were four themes regarding students’ experiences using HMD, including physiological (headache, motion sickness, and nausea), psychological (nervousness, uncertainty of what to expect in VR), personal abilities (first time VR user, not skilled with VR technology), and hardware quality (lag, latency, blurry screen, difficult to put on). Experiences in Scale Worlds had two themes, which were positive sentiment (interesting, enjoyable, exciting) and positive utility (visualization of extreme sizes, collaboration with partners, experience the impossible). Students reported would recommend VR to help learning.

Teachers’ Perception

Teachers reported experiences of using VR and perception of students’ using VR being a positive experience (engaging, enjoyable) and technology with high educational potential (VR as growing technology, better support for tech savvy generation, VR is possibly more awe-inducing, students to create their own VR world, integrate VR as part of the actual education system). There were two themes regarding Scale Worlds as part of a lesson, which were demand for more and variable content (less exciting graphics compared to games, want more details such as chemical reactions animated) and desire for greater usability (unclear instructions, insufficient interactive features). Regarding comparison of Scale Worlds to regular class period and implementation of VR, there were two themes: content design compatibility (greater integration of Scale Worlds with the actual class material) and resource allocation (time limit, how to best split students into groups, how many HMDs realistically are needed).

Discussion

Through an in-field study in a public middle school, this study obtained valuable and practical feedback regarding students’ and teachers’ experiences and perception regarding an immersive learning environment (developed based on education theory and went through usability evaluation) and feedback regarding VR technology for classroom use. As the primary user of in-class VR technology, the students reported experiencing VR-induced physiological discomfort. The specific discomforts reported were headache, motion sickness, and nausea, which have been reported in the literature (Jensen & Konradsen, 2018). Not all students reported the same symptoms and the occurrence of discomfort differed across students, and the symptoms subsided quickly after removing the HMD. It would be necessary for instructors to provide general safety precautions prior to using VR technology and institute regular rest break reminders throughout the class period to reduce the potential occurrence of discomfort. It may be possible to programmatically include reminders within the VR app to suggest time for rest breaks. It was unclear how frequently rest breaks should be suggested given the smaller sample, but it might be possible to start with every 5 min with gentle reminders as part of future extended usability evaluation. The authors do not assume VR technologies are fundamentally usable, but rather VR technologies present possibilities for different educational experiences.

It was interesting to learn that students were concerned about them being first time VR users not skilled with VR technology. It was speculated that students could be concerned about their performance using VR technology, as opposed to treating the HMD as a game or something they would not care as much because it was different from their standard education. It would be meaningful to pose follow-up questions in the future study to understand students attitude and how serious do they treat VR technology as part of classroom education.

Aside from the HMD itself, Scale Worlds was positively received as students reported the experience to be interesting, enjoyable, and supported the understanding of size and scale. It was observed that students took advantage of the virtual space to visualize entities that vastly differed in size. It was encouraging to know that students perceived Scale Worlds to be interesting, enjoyable, and exciting. There is a possibility that this was a “novelty effect” where students could use VR in a classroom. It would require a longer study (e.g., several weeks) and further collaboration with the teachers to assess whether students were excited because of the new technology. Despite the potential novelty effect, students did report that they utilized Scale Worlds to support their visualization of different entities and saw how big or small was relative to another. Working in teams also enabled student collaborations; it was observed that students who were not wearing the HMD (watched casting) attempted to guide the student who was wearing the HMD to complete the instructional activities. This suggested that learning activities could be designed to enhance collaboration among students even if they were not all wearing HMDs. Students may engage in Scale Worlds exploration while watching casted on a tablet, and thereby allowing those susceptible to VR discomfort still participate in the scaling experience.

The teachers focused on the content of SW and deployment of VR for classroom. They were the driver behind the administration of VR learning experience, and therefore the preparation and actual time and VR technology allocation were of their primary concern. Although the commonality of size and scale runs through a variety of STEM subjects, the teachers intended to deliver more targeted and focused topics within the subject of science, specifically energy and natural gas. On the other hand, the teachers recognized that the students enjoyed the experience and were engaged in learning. For future use cases, more specific, dynamic content of Scale Worlds may be included to address the desired experience from the teachers. For instance, a teacher mentioned “gamification” as they believed this could keep the students engaged in the long run. Since the research team provided the HMDs, the teachers were not immediately concerned of access to technology. For practical purposes, a group size of three students was suitable for the Scale Worlds activities. This has influenced the planning and subsequent follow up visits to the middle school. However, this might be a discussion with the middle school administrators if they intended to keep HMDs regularly accessible to their students. Overall, the teachers appreciated the content but desired for more elaborated educational content that matched their curriculum.

To address the limitations from the current visit, there was additional planning from the teachers’ end and more work. This finding highlights the importance of in-field studies because it was assumed that Scale Worlds was a usable immersive learning environment, yet in fact the actual deployment required more deliberate planning. This was the first in-field study at the middle school that limited to studying eight graders. Another visit has been planned (May 2024), with a potential of describing the unique human factors and interactions at an educational setting. A larger sample would be desirable.

Conclusion

This research forges new ground in HF/E research through in-field (classroom) study. Implementing VR technology as part of a class period required collaboration and co-design of content with the teachers who administered the instructional content. As immediate users, students directly experienced some discomfort associated with using HMDs. However, they still took advantage of visualization, experience the impossible, and teamwork to fully leverage Scale Worlds. Teachers’ were concerned usage and perception of the learning content, and utility, integration of VR, and deployment for classroom. Conducting in-field studies have extensive value as it provided practical feedback to develop future iteration of classroom VR technology.