Abstract
This article examines embodied interaction in a virtual reality learning environment. Studies of embodied interaction in immersive learning environments, like virtual reality, tend to treat all bodies the same without considering the nuanced cultural histories those bodies have with being mobile, especially within—and beyond—technology-mediated environments. In response, this study pivots from perspectives on embodied interaction that underscore the inextricable link between mind and body in favor of sociocultural perspectives to embodiment that emphasize the cultural-historical production of embodied interaction across space and over time. Through multimodal analysis of 10 learners’ experiences in a virtual reality experience called Thought for Food, this article contributes (1) an overt focus on the importance of feeling histories—embodied ways of sensing, feeling, and moving within digital environments—of learners engaging in virtual reality environments in order to promote equitable learning opportunities and (2) argues for future designs that are attuned to frictions—contestations between bodies and interfaces—that potentially collide with learners’ feeling histories.
Introduction
Virtual reality (VR) has been touted as a means to revolutionize educational possibilities, with early adopters celebrating its potential to connect learners with remote experts, imaginative locales, impossible interactions and more (Greenwald et al., 2017). Innovative approaches to VR in educational settings range from engaging learners in neurosurgery simulations (Pelargos et al., 2017) to the collaborative interaction with prehistoric petroglyphs (Kulik et al., 2018), among others. But VR experiences stretch across a vast experiential spectrum, from relatively passive 360-degree video experiences, like those featured in forms of immersive journalism (Jones, 2017) to increasingly complex experiences, like forms of collaborative gameplay. There is a need, therefore, to understand how users move and engage with learning material across this spectrum of experiences, including how they leverage previous embodied, lived experiences to propel themselves through designed VR experiences.
Indeed, the spectrum of VR experiences documented in the literature is expansive—and continues to evolve. Recent articles, for example, explore how VR can enhance fire safety behavior skills (Çakiroğlu and Gökoğlu, 2019), speed reading (Rau et al., 2018), evacuation training (Feng et al., 2018), and emotional skills of learners with autism spectrum disorder (Ip et al., 2018). Regardless of subject matter, however, VR experiences demand powerful forms of embodied interaction. For instance, Lindgren et al. (2016a) targeted embodied interaction as a critical component within a mixed reality simulation on students’ Science, Technology, Engineering, and Mathematics (STEM) learning, finding that the experience of whole-body activity significantly impacted learning gains for students, especially in terms of shifting student attitudes toward Science. Although their focus is on STEM learning, Lindgren and colleagues forefront the role of the body within immersive learning experiences, like virtual and mixed reality. Indeed, this focus on embodiment has been critical to a productive and influential line of research that has argued for “personalized educational experiences grounded in the learning affordances of human perception and bodily action” (Lindgren et al., 2016b). This article compliments others that have analyzed the relationship between embodied interaction and immersive learning experiences. However, studies of embodied interaction tend to treat all bodies the same without considering the nuanced cultural histories those bodies have with being mobile, especially within—and beyond—technology-mediated environments. Thus, in response, this study pivots from perspectives on embodied cognition that underscore the inextricable link between mind and body (Wilson, 2002) in favor of sociocultural perspectives to embodiment that emphasize the cultural-historical production of embodied interaction across space and over time (Ma, 2016, 2017; Taylor and Hall, 2013). In particular, this article contributes (1) an overt focus on the importance of feeling histories—embodied ways of sensing, feeling, and moving within digital environments—of learners engaging in VR environments in order to promote equitable learning opportunities and argues for (2) future designs that are attuned to potential frictions—defined as “practical, affective and emotional contestations between bodies and interfaces” (Ash et al., 2018a: 1140)—for learners. Our discussion brings our focus on feeling histories and frictions into contact with research on culturally responsive technologies and argues for immersive learning experiences that are “continuously and dynamically crafted through collaborative processes” (Srinivasan, 2018: 7).
The design experiment featured in this article describes learner engagement in a VR experience called “Thought for Food” (TFF). TFF introduces learners to issues related to food sustainability, particularly towards promoting sustainability identities by (re)orienting learners’ actions, interests, and identifications related to food sustainability (Polman and Hope, 2014). This article reports on one facet of this project, focusing particularly on the embodied “know-how” (Lemke et al., 2015) that VR environments demand from learners in order to enable learners equitable access to them. Entry into various elements of a VR experience, we argue, requires that users have specific feeling histories in order to effectively engage with the environment. Concerned about how feeling histories affect learners’ abilities to enter into and engage with VR experiences equitably, this article addresses the following, overlapping, research questions:
What feeling histories do learners bring with them into VR experience? How do those feeling histories impact their VR experience?
In the following sections, we first expand our working definition of feeling history, with a particular emphasis on the somatic qualities of interaction in digital environments. Then, we build from a focus on embodied interaction in digital settings with an emphasis on how that interaction is modulated and manipulated through the design of digital interfaces, particularly as a means to avoid frictions. From there, we provide further background on the design of TFF and our methods of data collection and analysis before turning to an explicit focus on the feeling histories that learners brought with them into their experience of TFF and the frictions they encountered. Finally, we conclude with a discussion that argues for designed VR experiences that are sensitive to the diverse feeling histories that learners carry with them in—and out—of the experience.
Theoretical orientation
This article examines how feeling histories inform learners’ experiences within designed VR settings. Histories, Cvetkovich (2003) writes, can “‘touch’ one another. There are resonant juxtapositions between past and present whose explanatory power is not causal or teleological; instead, the affective charge of investment, of being ‘touched’, brings the past forward into the present” (49).
This take on how history is felt as the past commingles with the present is comparable to Hansen’s (2004) description of how bodies retain previous experiences, or “how the past remains with the living present” (603). The “thickness” of the present, Hansen writes, “expands to include the entirety of the past in a retentional chain” (604). Embodied interaction, then, does not—and cannot—simply concern moving bodies working toward a learning goal, for example, but must consider how those bodies are both “time constituting and time constituted” (604).
Learners enter into VR experiences with their own histories of feeling which, in turn, affect how the body moves and feels in moments of direct perception: past histories with digital interfaces, like video game controllers and mobile devices, “touch” the present. Lemke (2013) provides further nuance to how the past is implicated in the present by describing how feeling histories accrue over time as “meaning-and-feeling continuities” (65). Focusing specifically on how feeling histories are produced in tandem with the material environment, Lemke argues that meaning-and-feeling continuities develop as bodies feel and interact in material environments, where meaning-making is understood as distributed “between organisms and environments, subjects and objects, cooperating persons and mediating artifacts” (75).
This perspective on feeling histories, especially its distribution across a digital environment, resonates with ethological approaches to the study of interaction between users and digital interfaces. Ethological perspectives underscore the somatic qualities of digital interactions (Ash, 2010) highlighting, for instance, how bodies are distributed “into the environment on the screen” (427). From this perspective, interaction with digital interfaces leads to a “somatic re-organization of the human being” (427).
Ash et al. (2018a, 2018b) have expanded this take on ethological perspectives, offering fine-grained accounts of the diverse ways users engage with (and are engaged by) digital interfaces. They offer unit, vibration, and tone as post-phenomenological approaches for understanding interaction with digital interfaces. Broadly, Ash et al. (2018b) explore the affective nature of “design elements,” like buttons, menus, and checkout pages, focusing on how those elements involve bodily modulations and modifications (167). More specifically, they break down each of these elements into discrete units that each have the power to modulate and modify how a user’s body engages with it. Importantly, just as the user manipulates the interface, the interface—via these units—manipulates the user. Units communicate with users in ways that “prime and shape various responses and actions on a series of material and habitual levels” (173). Through this shaping of responses, interfaces produce frictions.
Recent related research across disciplines from human computer interaction to human geography has begun to question the “frictions” that occur as users interact with various interfaces. As Galloway (2012) writes, an interface is “an autonomous zone of interaction … concerned as much with unworkability and obfuscation as with connectivity and transparency” (120). Not all interactions with interfaces are smooth. As Rose (2016) notes, any kind of “obfuscation” becomes even more apparent given a pervasive mobile media culture in which most users’ interactions with interfaces are, for example, facilitated by fingers touching, tapping, and swiping across glossy screens produced by the contemporary economy of aesthetics. Still, frictions are “inherent to interfaces” (Rose, 2016: 343). They may act as gate-keeping mechanisms, like password screens, but they may “also act as sites of grip, encouraging someone to continue using or engaging with that interface because of the contestation faced by the user, such as when learning to play a videogame” (Ash et al., 2018a: 1140). Frictions, then, are “practical, affective and emotional contestations between bodies and interfaces” (1140) that can prompt—in the context of VR—both encouragement and discouragement. Moreover, they signify “bodily and technical obstacles or hesitancies that interrupt, slow or stop a user from completing a task within a digital interface” (1138).
Frictions are intimately related to thresholds. Thresholds are “key points in an interface that need to be passed through” (Ash et al., 2018a: 1138). Examples of thresholds in familiar digital interfaces include password screens that require text entry, buttons that users might click to submit a document, or even proving that a user is not a bot by clicking pictures of vehicles. Thresholds and frictions, then, occur simultaneously, although some thresholds contain more friction than others. Entering a single password, one could argue, is an easier task than proving that one is not a bot and thus produces less friction for a user.
In the following, we bring this focus on feeling histories and frictions into contact with contextual information regarding our designed VR experience: TTF. We then turn toward fuller portraits of our focal learners—Dale, Maria, and Emile—and the multimodal interaction analysis (MIA) we performed in order to address our research questions related to how their feeling histories impacted their experience in TFF.
Method
Thought for food
TFF is a VR experience designed to promote sustainable identities for participants. Informed by sociocultural perspectives of learning (Lave and Wenger, 1991), a driving goal of TFF is to change the quality of participation in practices embedded within a specific community of practice. Specifically, and inspired by Polman and Hope (2014), TFF seeks to promote shifts in three focal areas related to food sustainability: Actions, or the how learners can actually engage with issues related to food sustainability; Interests, or “openness and stance towards aspects of action” related to food sustainability across time and space; and Identifications, “or ways that actions connect to people’s identity affiliations, in the past, present, and future” (317). While these tenets determined the ethos of TFF, this article reports on learner interaction within the designed space itself, particularly the “frictions” that learners encountered throughout their experience and how those frictions related to learners’ feeling histories.
Sensitive to the spectrum of interactions possible within VR environments (Greenwald et al., 2017) and recognizing that that spectrum consists of more than those that we integrated into our design, TFF includes four unique environments, with each environment enabling learners to immerse themselves in different forms of embodied interaction related to food production, including:
Any Market, USA: a tutorial zone that introduces learners to the mechanics of VR and TFF, and enables learners to transport to other environments. By picking up various objects—chicken breasts, bananas, and a jar of goat’s milk—learners teleport to other settings. The Boxing of the Chicken: an environment that positions the learner as a chicken living in both cage and cage-free habitats. This environment demands that learners, playing the role of chicken, effectively navigate a cage—or a field—in order to obtain food. The Extravagance of the Banana: an environment which forces learners to actively move bananas from tree to table, including various forms of user-initiated transportation, including boat, train, and plane. This environment demands that learners actively move their bodies—including rowing virtual oars, pumping virtual train cars, or flying virtual wings—to propel the carton of bananas forward. In Every Drop of Goat’s Milk: A 360-degree video-based environment that immerses learners within the daily activities of a local goat farm, including feeding, nail trimming, and cheese packaging. This environment is passive, when compared to others, only requiring the learner to shift their visual field in order to view different aspects of the environment.
While each environment was engineered to produce different affective intensities for learners—including claustrophobia, for example, in the case of our chicken-focused environment, or physical fatigue, in the case of our banana-focused environment—analysis reveals that these diverse environments recruited various feeling histories that our design did not necessarily account for. As we report here, these feeling histories interfered with the overall experience of learners throughout their experience at best and (re)produced forms of exclusion at worst.
This study of feeling histories and frictions in VR is a part of a larger design experiment (Cobb et al., 2003) intended to engineer innovative learning opportunities related to issues of food sustainability in VR environments. This design experiment of VR, housed at a large, public university in the Northeast US, draws on the experiences of 10 learners, including undergraduate and graduate students, as well as adults working for the university’s sustainability institute. The design of the experience was continually refined based on learner feedback as well as our initial analyses, thus fueling subsequent iterations by the design team.
Learners
In this article, we focus primarily on the experiences of three learners: Dale, Maria, and Emile (Table 1, all names pseudonyms).
Focal learners.
We focus on these three learners because they provide a unique cross-section of those who experienced TFF. All three learners, as Table 1 illustrates, are involved in local sustainability efforts. This desire to provide alternative ways of teaching others about sustainability issues spurred both their experience as well as the kind of feedback they provided us: They were acutely sensitive to design decisions and the possible impact those decisions could have on others. Dale, for instance, was incredibly enthusiastic about the possibility of using VR to teach others about issues like climate change or food consumption or trash production. Maria, in contrast, was quite skeptical, unsure of how those kinds of topics could be broached in a way that was not didactic. Unlike Dale and Emile, she also did not have a history playing video games—a feeling history that, we argue, can serve to ease frictions in VR, and that we discuss further below. She did describe previous experiences watching 360-degree videos on YouTube; however, and thus felt comfortable having embodied, although not necessarily VR, media experiences. Emile was fairly neutral about the use of VR for the development of sustainable identities, but did note that he was excited by the role that new technologies could provide for immersive experiences in localities around the world.
Virtual reality lab
The VR experience was housed in a lab space dedicated to the exploration of teaching and learning with technology. Upon their arrival at the lab, we briefly met with individual learners to inform them about the VR experience that they would encounter. These brief meetings primarily focused on our reasoning for creating TFF rather than the mechanics with which learners would engage or what they would encounter. Once learners were ready to engage, they entered into the VR arena (Figure 1) and were given brief verbal directions on how to use the HTC Vive controller while one member of the team placed the VR goggles and headphones on the learner’s head.
The virtual reality arena.
Once the learner was ready to begin, the VR environment initiated and they were transported “inside” the first designed experience, the tutorial zone Any Market, USA.
Data collection
We used two video cameras, one roaming alongside a learner and one on a tripod in the corner of the VR space, to document all user activity. Learners also wore a wireless lapel mic for audio capture. Learners often narrated their experience—often ranging from 20 to 30 minutes—providing a running commentary of their experience. Researchers documented all VR interactions, taking field notes throughout the experience. After learners completed the experience, or chose to stop, we conducted semi-structured interviews with them, which lasted approximately 15–30 minutes. These semi-structured interviews asked, broadly, about their experience and, inevitably, led to participants wanting to discuss the frictions that they encountered—as if those moments had been inscribed upon their bodies through the VR experience. Thus, these semi-structured interviews primed our research team to explore frictional moments further through our analysis.
Data analysis
Analytically, in-the-moment observations, re-viewing of video data, and interviews frequently called attention to the importance of “frictions” (Ash et al., 2018a) and their relationship to particular moments within the TFF environment. As we have described earlier, frictions signify a series of “bodily and technical obstacles or hesitancies that interrupt, slow or stop a user from completing a task within a digital interface” (3). Our analytic process led us to re-visit all captured data in order to document frictional moments that learners encountered.
With a focus on frictions, we then performed MIA (Jewitt, 2009), which explicitly targeted embodied movement across micro-/macro-scales, including hand gestures (micro), body movement (macro), and talk, as well as what the learner what engaging with in the VR environment.
Multimodal interaction analysis
MIA has been deployed by others (Jewitt, 2009; Sakr et al., 2014) to provide a set of resources to analyze interactions that spread across physical and digital spaces. This kind of method is critically important as physical bodies are increasingly “re-mapped” through their interaction with digital interfaces (Jewitt, 2009). A multimodal approach, Jewitt (2009) writes, provides the methodological capacity to trace how physical-digital technologies generate “new forms of interaction and enable new action, physical, perceptual, and bodily experiences” (258). The purpose of this level of analysis in this article was to develop an understanding of the nuances of the frictions learners encountered. For instance, some frictions were produced through designed components of the environment, like a tree, while others were produced by the tools used by learners, like the handheld VR controller. Figure 2 illustrates what this analysis looked like.
Multimodal transcription example.
Following tenets of multimodal methods for researching digital technologies, we analyzed our data through a “systematic description of modes and their semiotic resources” (Jewitt, 2009). Specifically, we targeted multimodal ensembles that were most salient when learners engaged with the VR environment. As Jewitt (2009) writes, multimodal ensembles are “representations or interactions that consist of more than one mode” and can “therefore be seen as a material outcome or trace of the social context, available modes and modal affordances, the technology available and the agency of an individual” (259). Thus, the modes of communication that came together throughout the VR experience—and thus our analytic foci—include finger movements/gestures, body movement, speech, observer re-direction, and VR interaction. As evidenced in the example, finger movements included actions like tapping trigger buttons on the controller. A focus on body movement, visible through the still images in the transcript, enabled us to witness how learners moved within the VR environment, as well as what they were moved by. In Figure 2 for instance, a learner engages and reads a directions sign, tilting and turning her head in order to read it more clearly. Speech was also often part of the multimodal ensemble as learners articulated their actions, or difficulties, or successes to observers. A focus on observer re-direction, then, signifies moments in which observers—the designer-researchers—felt the need to prompt learners in order to help them navigate within the VR setting. Finally, a focus on VR interaction shifted our analytic lens toward the specific elements within the VR experience that learners were engaging with, like the “pink circle” that the learner is trying to touch in Figure 2 Analyzing these multimodal ensembles, then, enabled us to identify the frictions learners encountered throughout TFF. Moreover, we were further able to identity which “mode”—i.e. finger gesture, body movement—was the primary cause of that friction. Our findings, then, describe frictions across three scales of VR experience: (1) the micro-scale, like frictions pertaining to the hand movements; (2) macro-scale frictions, like frictions related to the whole body; and (3) how these frictions accrue over time for learners and how the accumulation of frictions impact learner experiences. Our discussion brings these focal areas together and centers on the relationship between frictions, culturally responsive technologies and designing for justice.
Findings: “I Felt Embarrassed:” Feeling histories and frictions from hand to body
Handheld frictions
The handheld controller participants used to move through the experience often took time to get accustomed to. Participants repeatedly reported that they had previous experiences with other handheld devices, like video game controllers, or even other VR handhelds. While this background is not surprising, we argue that these previous controller-based feeling histories took critical time away from their initial moments in TFF to overcome, often starting the experience off with a sense of “frustration” that, we argue, bled over to the rest of the experience. As one learner, Maria, states: … the first two to three minutes were quite annoying. Because I did not get how to control the controller. I know you taught me how to control it, but I mastered the control through [a later part of the experience]. Before that, I was just pressing buttons.
Six possible hand combinations with HTC Vive.
While it is important to note that the learner felt that she mastered the button combinations in a later part of the experience, we are most concerned by these initial moments of annoyance incited by the friction she encountered, frictions that were, initially, instigated by an inability to grip the controller, tap specific buttons, and coordinate multiple button presses. For Maria, this was a frustrating experience that, as we report later, filtered into later parts of her experience as well. Furthermore, and importantly, this is a friction encountered at the scale of the hand—thus only one “mode” as identified in our multimodal transcript. Frictions, we argue in the following section, become more complicated as more elements of the “multimodal ensemble” become entangled—particularly the moving body.
Body frictions
This sense of feeling history-generated frustration extended to other parts of TFF for Maria and others. In a later portion of the experience, learners were tasked to row a boat for a brief period of time to transport bananas (Figure 4).
First-person view of rowing.
We did not account for the feeling history of rowing; however, including how one should position their body, how the arms must move back and forth, or at what pace one’s arms should even move. For some learners, like, Dale, this task took between 3 and 5 minutes which was nearly 25% of their experience (Figure 5).
Multimodal transcription of Dale orienting himself to the rowboat.
Multimodal transcription of Emile trying to row in the boat.
Our multimodal transcript evidences Dale’s frustration in multiple ways. First, he must coordinate his fingers, tapping the button on the controller to grasp the virtual oars as he picks them up. Then, he must orient his body in a way that is comfortable to begin to row the boat. Part of Dale’s troubles stem from the fact that he is standing—all other participants chose to sit in this part of the experience. Because he stands, Dale’s arms are forced to move back-and-forth in a way that does not feel “natural” to him. That is, his arms remain rigid, straight as he pulls them back, rather than angled, as they would be when rowing a boat.
Others encountered similar frictions when rowing. Emile, another learner who had difficulty with this task, reported: When I had to row. It was pretty hard. I felt sore little bit. I think it is because I don’t know how to row. I was actually sweating.
In order to row the boat, he must simultaneously use both oars. The transcript reveals, however, that Emile is largely concerned with his left hand—the left oar—rather than coordinating both arms. Rowing, then, requires that Emile orient his body in a particular way, move his arms in a particular way, and simultaneously tap the controller buttons in a particular way at a particular time: Hand frictions collide with body frictions. Later, Emile focuses on his right hand—forgetting the left—which adds to the difficulties. All in all, it takes him nearly 5 minutes to move through the rowing portion of the experience.
Rowing, of course, was unique to one facet of TFF. While, on the one hand, the design was meant to challenge learners in order to make them “feel” the difficulties of transporting food across the globe, it also, on the other hand, made us more sensitive to potential issues that result from challenging moments early on in an immersive experience like the one presented here. Because rowing the boat was required, by design, to enable participants to move on to a subsequent phase, Emile enthusiastically worked to figure out how to efficiently do it, which took a substantial amount of time and therefore impacted his later experience. We further elaborate on this reverberation of frictions in the next section.
The reverberations of frictions: How frictions persist over time
As a result of learner feedback in earlier iterations of TFF, Dale, Emile, and Maria were able to experience Any Market, USA, a “tutorial zone” that acted as an initial entry-point to the experience for learners. It included objects learners would “pick up” (i.e. chicken breast), which would subsequently transport the learner to an experience centered on sustainability issues related to that object (i.e. chickens living in cramped quarters). One learner noted, for instance, that she felt “embarrassed” when she started off in the market. “I didn’t have any direction like what to do,” she said. “I thought it didn’t start, that the experience had not started yet. I just looked around.”
Responding to this feedback, the design team integrated signs into the market, teaching learners how to squeeze buttons on the controller and how to teleport from one location to another (Figure 7).
“Any Market, USA,” entry-point into TFF, including teleports to other food-oriented “worlds”.
Still, because of learners’ previous feeling histories, some subsequent learners recognized the teleportation sign as a tutorial but were unable to figure out how to engage with it effectively. In Figure 7, for instance, a pink checkmark appears above the sign once the learner has successfully learned to teleport. Multiple learners continued to engage with the checkmark, believing they had to manipulate it in some way. This entry-point into the TFF experience, then, often resulted in feelings of embarrassment, as evidenced above, or slight frustration, or left learners with a general sense of uncertainty. It is these feelings that this section is most concerned with, particularly as they potentially inhibit learners from engaging with the virtual environment in ways that account for their historical ways of feeling and that propel them forward with affective encounters that can capitalize on the deeply immersive nature of VR.
Maria’s entry into TFF, for example, does not start well. Upon arriving in the market, Maria first looks around to observe her surroundings, before trying to interact with the sign. She asks the designers: “Is that in?” with the belief that she needed to “throw” the pink line signifying that she was teleporting into a circle above the yellow sign. In fact, the pink line signaled where her avatar would teleport to when she pressed the appropriate button combination. This friction, then, stems from the (mis)reading of signs, of interpreting them in ways that designers did not necessarily intend. Moreover, Maria’s inability to “read” the teleport line; however, relates back to an absent feeling history: She has not previously experienced “fast travel”—as it is referred to in video games—in which one does not have to “physically” move from one area of a setting to another. Part of her frustration, then, results from the constant reading and mis-reading of signs—including literal signs designed to help her, as well as those that demand particular feeling histories, like those that develop through playing video games.
Maria’s experience led us to further analyze the amplification of frictions over time. In a later part of the experience, the 360-degree video that immerses the learner in a local goat farm, Maria, again, encountered a friction—but one that was intimately connected to her previous (mis)reading of navigational signs in the Market (Figure 8). Because of her previous experience in the market, Maria believes that there will be signs to help her move throughout the setting. She also believes that she can interact with the goats and with the farmer. Because this is a 360-degree video—and thus a relatively passive viewing experience in which the learner can look around the entire space but not engage with any parts of it—she is confused.
Maria encounters friction in the market and is unable to progress.
As depicted in Figure 9 Maria first tries to engage with the scene with her right hand and then shifts to her left hand before trying both hands at one time. Because instructions guided her in previous instances, Maria then begins to turn around in place. In her interview, she told us that she was looking for signs that would help her figure out how to appropriately interact with the goats and the farmer. We consider this moment to be an amplification of Maria’s earlier friction-generated frustration, particularly one that led her to disengage from the experience overall. In an ironic reversal, in this instance, we—as designers—produced a feeling history for Maria earlier in the experience, priming her to read signs and engage with instructions, before stripping that away from her.
Maria attempting to interact with the goats and farmer.
Henriques (2014) defines amplification as a kind of intensity and that “these intensities are expressed across areas and volumes, as pressure, force, effort, heat or auditory volume” (84). Our focus on the amplitude of frictions, in Maria’s case, allows us to consider the relationship among the frictions, particularly as they were “expressed across areas”. Frictions, in immersive settings, like VR, are not isolated instances. Maria, for instance, encountered 12 different frictions throughout her experience. She worked her way through those frictions for a total of 8 minutes—out of a 16-minute VR session.
While it is easy for us to reflect on the design decisions that we made, we are—now—just as sensitive to the fact that we did not account for the diverse feeling histories of our learners, ranging from the scale of the hand, to the scale of the body, and, as evident in this section, how frictions can accumulate throughout one’s experience in an immersive environment. We further explore reflections on what it means to design immersive settings with learners, particularly as a means to engage more deeply the feeling histories that they carry with them in our discussion.
Discussion: Co-designing culturally responsive immersive experiences to account for feeling histories
In a review of research of the changing social spaces of learning—both physical and digital—Leander et al. (2010) caution readers that learning settings are increasingly mobile, that they “will not hold still” (385). Immersive technologies, like virtual and augmented reality, are a harbinger of increasingly embodied, mobile digital technologies and experiences. This article is less concerned with, say, how to teach STEM through VR than it is in how to understand how bodies move and feel within learning environments that refuse to “hold still,” that demand bodies to move—or row, or fly, or climb—in ways that they typically do not in a traditional, often sedentary, learning settings. Thus, a concern for this article revolves around the attention given to the moving, active body in current educational research, especially how bodies sense, feel, and propel themselves through immersive, embodied environments.
Our focus on feeling histories and frictions, however, cautions against celebrating movement simply because it contrasts with forms of sedentary, immobile learning. Rather, it calls for an acute attunement by researchers and designers not only to how bodies move within these settings, but also to the ways of being mobile that learners carry with them into those settings. When researchers design for movement within immersive settings like VR what ways of moving and feeling are privileged? What ways of moving and feeling are cut out?
One way forward toward the design of more equitable VR learning experiences hinges on creating culturally responsive learning technologies that respond to participants’ ways of feeling and moving. Bang et al. (2013) refer to this process as producing technologies that are “dislodged from colonial legacies that implicitly or explicitly position technologies as having only Western-European ontologies” (708). While the research presented here does not focus on learners from non-dominant communities necessarily, we repeatedly witnessed early warning signs of potentially re-producing Western-European ontologies through the ways in which learners were able—or unable—to move and feel their way through the experience. By targeting moments of friction, we were able to begin to identify instances within TFF that privileged specific ways of being and moving. The instance we described in which Maria was able to effectively manipulate the controller, for example, contrasted greatly with other learners who seamlessly began moving, teleporting, and grabbing various objects. The controls used in VR settings tend to follow a “one-sized fits all” approach. Despite all of the agency a VR setting might try to afford a learner, the learners first experience with the environment—through the controller—leaves them with no choice and, inadvertently, positions them in potentially confusing or “embarrassing”—as in our example—situations.
Another way forward includes the design of VR experiences that are “continuously and dynamically crafted through collaborative processes” (Srinivasan, 2018: 7). This approach to the production of VR experiences pivots from top-down, epistemologically saturated experiences to those that are co-designed alongside the learners—and the communities—who will participate in them. What might VR experiences look—and feel like—that “support a range of practices, visions, priorities, and belief systems of indigenous and non-Western cultures” (7)? While a team of researchers and experts in video game and VR design worked together to produce TFF, the experience was deliberately left open for future expansion to enable immersion within other “worlds” related to food sustainability. Alongside foci like chicken, goats, and bananas, for instance, we imagined future experiences related to, say, plastic bag waste and water consumption. TFF, then, would not only be collaboratively produced by various stakeholders, but could also be the kind of “continuous” design of which Srinivasan writes. By being continuous, then, VR experiences could be renewed through “collaborative processes,” rather than one-and-done encounters. Rather than being designed from “cradle to grave,” as Braungart and McDonough (2002) write, they might be designed from “cradle to cradle”—continuously emerging as something new alongside those who will immerse themselves within them. We recognize that a progressive approach to VR is a challenge. Yet, we still wonder: To what degree can youth, educators, and other learners customize and transform immersive technologies for learning to meet their needs and goals over time?
Culturally responsive immersive learning settings, we argue, must be sensitive to time. This does not necessarily mean more time in VR, for example, but what we think of as flexible experiential temporalities. The experience of time in VR has been the subject in literature on immersive experiences recently—and is one that demands more attention. Rau et al. (2018), for example, compared and contrasted user’s experience speed reading in VR versus their experience speed reading on a desktop computer. Because it takes longer for learners to read within immersive reality settings, they argue that “[d]esigners of VR and AR programs should give users about 10% more time to respond to text-processing tasks, compared with those programs for PC, to make the difficulty equivalent” (244). A focus on feeling histories provides further nuance to the experience of time, particularly in immersive learning settings like VR and AR. The experience of time in technology-enhanced settings is multi-dimensional, however. And this is a critical point: 10% more time is not experienced the same by all learners. As Ligorio and Ritella (2010) demonstrate, time is felt. It is experienced in different ways, by different learners, and “enters into social-psychological phenomena at multiple levels, in different functional roles, and as different types of processes” (434). Moreover, time is felt differently based on the feeling histories that learners bring within them—both in and out of immersive settings. Focusing on the experience of time by Indigenous children, for instance, Bang et al. (2013) describe how learning settings are typically developed from “colonial timeframes and not Indigenous timeframes of place” (724). There are three overlapping perspectives on time here that are important for the design and implementation of VR settings. We put them forth as questions to consider: (1) how long does an aspect of the VR experience take (i.e. 2 minutes, 8 minutes?); (2) how does the time spent performing an aspect of the VR experience feel (i.e. long and arduous, quick and rapid); (3) whose feelings of time are privileged in the design (i.e. Designers? Teachers? Western-European learners?). Addressing these questions, and noting their nuances, can lead toward immersive learning experiences that provide flexible experiential temporalities for learners. Creating diverse temporal pathways, we argue, can lead toward co-designs that are more equitable and just.
Finally, by focusing on the feeling histories of learners engaging in VR, designers may more readily align to principles put forth for Design Justice (Costanza-Chock, 2018). Design Justice questions who gets to participate in design, who those designs are produced for (or with), and what values designers encode into designed objects and systems (11). Design Justice shifts designers—as well as researchers, educators, and learners who design together—away from what Latour refers to as “matters of concerns” and toward what de la Bellacasa refers to as “matters of care”. Our design of TFF was imbued with concern: we centered climate change and sustainability. Shifting from concern to care, however, might have led us to a deeper consideration of the unique learners who would move and feel within our designed experience. As de la Bellacasa (2017) writes, [p]ositioning for care emerges as an oppositional practice that both creates trouble in the democratic assembly of articulate concerns as well as generates possibility: it reminds us of exclusion and suffering and fosters affective involvements with the becomings of science and technology. (26)
Statements on funding, open data, ethics, and conflict of interest
The data guiding this research are not publicly available due to its private nature. The research was approved and overseen by the Institutional Review Board at the institution where the research project took place. All participants provided informed consent prior to taking part in the study.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.