Abstract
Introduction
Entering college is an important developmental transition in the lives of young adults that involves a variety of new opportunities and challenges. Many first-year college students live independently from their parents for the first time and actively explore new friendships, social activities, and academic interests. Incoming students also may experience stressors related to homesickness, financial pressures, academic demands, time management, pressure to fit in socially, and other challenges.1–3 Research shows that health risk behaviors increase among first-year college students, including alcohol and substance misuse as well as violence. The first year of university is a particularly high-risk period for alcohol consumption, including binge drinking and heavy drinking. 4 As students spend time socializing to establish friendships and a social network after arriving at university, they typically attend dorm parties, fraternity or sorority gatherings, and bar events in a greater proportion than upperclassmen, 5 and alcohol and drug use are highly prevalent and socially approved in these environments. Furthermore, the prevalence of interpersonal, dating, or sexual violence on college campuses is high. 6 In a large study of university students 7 one in five college students reported at least one incident of sexual assault (e.g., unwanted touching, attempted or completed penetration); women (28%) and gender nonconforming students (38%) were disproportionately victimized, and men also reported being sexually assaulted (12.5%). Other studies show that victimization rates are high among first-year students, with up to 50% of campus sexual assaults occurring among students new to campus. 8 Alcohol and drug misuse, violence, and sexual violence among college students are associated with a host of adverse physical and mental health consequences including physical injuries, post-traumatic stress disorder (PTSD), depression, anxiety, and substance use disorder, all of which can interfere with one’s ability to succeed in school and with other personal, social, and occupational tasks and milestones. 9
Adapting evidence-based prevention programs for college youth
There is a dearth of tested and effective comprehensive primary prevention strategies available for substance use, violence, and sexual violence among college-age youth. To develop effective prevention programs, the field must consider where the most progress has been made. According to the White House Task Force to Protect Students from Sexual Assault ( 10 , page 5), new and adapted prevention strategies are needed that are “informed by theory and knowledge about the components or characteristics of effective prevention for other similar behaviors,” including evidence-based programs developed for younger age groups (i.e., secondary school students). Primary prevention efforts that address both substance misuse and violence are appropriate and necessary because violence victimization often occurs when the victim or perpetrator is under the influence of alcohol or drugs.6,7 Moreover, preventive interventions that target multiple health problems by addressing shared risk and protective factors offer the potential for providing the greatest public health benefit in the most efficient way. Shared risk factors for alcohol misuse, drug abuse, and violence include anxiety, depression, impulsivity, peer pressure, high perceived prevalence and acceptability of these behaviors, and adverse childhood experiences. Shared protective factors include strong coping skills, positive self-esteem, and involvement in close relationships and supportive communities.11,12
Life skills training
Life Skills Training (LST) is a leading evidence-based prevention program that addresses several shared risk and protective factors for drug misuse and violence. The LST program, initially designed and tested among secondary school youth, teaches personal self-management skills, social skills, and other life skills needed to successfully resist substance use/abuse and violence, navigate developmental tasks, increase resilience, and facilitate healthy psychosocial development. The LST approach is supported by scientific evidence published in a series of randomized controlled trials reported in over 35 peer reviewed journal articles involving 18 separate cohorts. 13 Findings from these studies show that LST is highly effective, typically reducing alcohol, tobacco, marijuana, and other illicit drug use by up to 50% or more. Studies have shown that the LST middle school program has long-term effects that last until the end of high school14,15 and into young adulthood. 16 In addition to effects on substance use, LST can effectively prevent verbal and physical aggression, fighting, and delinquency. 17 A recent study showed the LST program supplemented with an educational video game was effective in reducing bullying perpetration, 18 and a hybrid version of the LST program with e-learning modules and limited classroom sessions was effective in reducing substance use. 19
LST for college
The LST program was recently adapted for college-age youth to prevent substance use, violence, and related risk behaviors. One component of the LST for College program focused on managing change and conquering challenges. Students identified personal areas of challenge and learned how to manage and respond to these stressors. Students learned techniques for reducing stress and strategies to build resilience and overall well-being. They were taught a five-step problem-solving/decision-making method to develop healthy solutions and make healthy decisions. The program helped students understand how peers can affect this process and how to avoid undue peer influence. Another component of the program focused on fostering the skills needed to form healthy personal relationships. Students learned to identify both verbal and nonverbal cues to avoid misunderstandings and improve communication as well as guidelines for healthy social media use. Participants practiced the appropriate use of verbal assertiveness skills and how to respond to persuasive tactics to engage in unhealthy or improper behavior. They learned techniques to develop new personal relationships, recognize relationship boundaries, and handle situations in which boundaries are crossed. The program taught students skills for resolving conflict including the use of negotiation and compromise to achieve common ground. In another component of the program, students learned skills for healthy intimate relationships, including those involving sexual activity. Participants developed their understanding of how to form and maintain intimate relationships. Students were taught the characteristics of consent in sexual relationships and that consent requires voluntary, mutual, unambiguous, ongoing, verbal agreement. They practiced communication and assertiveness skills to request and give consent. They examined strategies to avoid high-risk drinking and substance use, including guidelines for the proper use of prescription and over-the-counter medications. Students practiced ways to respond to a variety of situations with active bystander techniques, including responding directly, distracting the perpetrator, and delegating responsibility and seeking help.
In a rigorous evaluation of the college adaptation of LST, the intervention model reduced sexual acts without consent in a randomized controlled trial of college-age youth. 20 The study was conducted at a military academy, and incoming students either received the adapted prevention program or served in the control group. Findings indicated that the program reduced sexual acts without consent by nearly half, with 4.4% in the intervention group versus 7.4% in the control group reporting this outcome. This study represents one of the first rigorous prevention trials of college-age students to demonstrate reductions in sexual violence.
Immersive digital technologies: opportunities to enhance intervention effects
Virtual reality (VR) technology is a simulation of a three-dimensional environment that enables the participant or learner to interact with people and objects in an immersive, detail-rich, and authentic seeming world in a subjectively realistic way. VR uses electronic equipment including a specially designed headset with immersive video and audio and handheld devices with motion sensors to enhance a sense of presence, where the learner engages in a virtual environment in which their movements are tracked and reflected back to them. A growing body of literature has examined the effectiveness of VR as a tool for addressing mental health issues and treating psychological disorders. The therapeutic potential of VR stems from its ability to combine the control and safety of a treatment or lab setting with the immersive realism of a natural environment. For example, VR can simulate various settings that may evoke a fearful reaction (e.g., heights, crowds, combat zones) but do so in a controlled and customized way. By allowing for gradual desensitization in a safe virtual environment, VR is an especially useful tool in the treatment of anxiety disorders, 21 phobias, 22 and PTSD. 23 The controlled, immersive environments produced by VR have also been used effectively for cue exposure therapies for addictive and eating disorders. VR simulated environments can expose individuals to various triggers and cues where cravings are elicited (e.g., bar situations, different food environments), providing opportunities to practice coping strategies to reduce cravings and to recognize and change maladaptive response patterns. VR approaches have been shown to be effective in reducing craving for different substances 24 and in reducing binge eating episodes and improving body image disturbances. 25 Finally, VR immersive scenarios can provide opportunities for practicing social skills in a safe and controlled environment, where users can engage in complex virtual social interactions that are scripted and customizable. For example, psychological treatments that use cognitive behavioral skills training techniques can use VR scenarios to provide practice opportunities for responding to social cues, such as recognizing and reacting appropriately to facial expressions and eye contact. Studies have shown that VR is an effective tool for improving emotion regulation in youth with high-functioning autism 26 and can reduce anger, improve conflict-resolution skills, and decrease aggression among youth with schizophrenia. 27
Given these findings, significant potential may exist in using VR technologies for other interventions that use cognitive behavioral skills training techniques. The most effective preventive interventions for substance misuse and violence among youth often use these skills training techniques to promote protective factors and reduce risk factors, and may benefit from incorporating VR technologies. VR immersive scenarios can expose youth to potentially challenging simulated social situations and provide opportunities to practice appropriate social and coping skills in a safe and controlled virtual scenario. VR may be particularly effective as a supplementary tool to provide virtual skills practice opportunities after participants have been exposed to evidence-based prevention content that teaches cognitive behavioral skills to promote resilience and new life skills.
Goal of the present study
The goal of the present study was to develop and pilot test VR modules to supplement an evidence-based drug abuse and violence prevention program for college students. The LST prevention program is based on a positive youth development approach that promotes anti-drug and anti-violence attitudes and perceived norms. The program teaches personal self-management and social competency skills to address skill deficits that have been linked to the etiology of adolescent substance use and violence. 28 A key component of the LST program is providing opportunities to practice the skills taught in the program. 13 In the present study, we developed and tested VR behavioral rehearsal scenarios in an immersive virtual setting, in which college students practiced identifying inappropriate interactions, resolving common conflicts, reinforcing positive norms, and resilience and prosocial skills, after completing a series of prevention e-learning modules. A multi-method approach was used to assess the impact and feasibility of the modules including pre-post surveys and post-intervention qualitative interviews.
Methods
Sample and recruitment
Participants in the present study included college students (N = 40) and educators (N = 5) from a public university in the northeastern US. Participating students were 18 or older (M = 19.9, SD = 1.3) and 78% lived on campus. Students reported their race/ethnicity as 65% White, 20% Black, 2.5% Asian, 2.5% Middle Eastern, 2.5% did not report their race, and 35% reported that they were Latino/Hispanic (students could select more than one race/ethnicity category). Regarding gender, 47.5% of students reported female, 32.5% male, 15% nonbinary, and 5% did not report their gender. Five students reported that they were transgender. In terms of sexual orientation, 40% reported that they were heterosexual, 27.5% bisexual, 10% gay, 12.5% queer, and 10% pansexual. About one-third of student respondents indicated that they had a disability. Faculty/staff participants were 60% male, 40% female.
Recruitment materials detailed the study’s purpose, participant involvement, and incentives ($10 gift card for students, and $25 gift card for educators). Students were recruited using flyers posted on campus and online, through announcements in psychology courses, and via information sent to students by email or social media platforms used by student organizations. Undergraduate university students who were at least 18 years old were eligible to participate. Written consent was obtained from participants interested in the study. They completed a brief confidential online demographics questionnaire, and an online pretest survey assessing life skills knowledge and anti-violence attitudes, which took approximately 5 min. The study protocol was approved by the Institutional Review Boards at the participating university and research and development company. The fieldwork for the study spanned from October to December 2021.
Procedure
Study design: intervention components and participant assessments.
Student participants
Study activities took place in-person in a dedicated on-campus computer laboratory. In these sessions, a single study participant was instructed by a trained research staff member to engage in a series of activities. First, the participant viewed three e-learning modules, each between 10 and 15 min in duration, that focused on Building Healthy Relationships, Being Assertive, and Protecting Yourself and Others (see Table 1 for the learning objectives of each module). These specific e-learning modules were selected from the larger LST for College program because they provided important and developmentally appropriate prevention content that was closely matched to the VR module interactive scenarios.
Next, the participant was instructed on how to engage with the VR module prototypes, which were each approximately 3 min in duration. The research staff member asked the participant if they had any previous experience using VR headsets; and if so, whether they had experienced any negative effects (e.g., motion sickness). In the absence of any concerns, the researcher assisted the study participant in mounting and operating the VR headset and handheld controllers (Meta Oculus Quest 2). The participant was given a brief orientation of the VR interface, including instructions on how to navigate through the immersive environment by moving their head and using the controllers, and how to select response options as they completed the branched scenarios presented in the module.
After completing each VR module, the participant provided written feedback on the relevance and appeal of that specific module. After viewing all three VR modules, the participant was asked to complete the posttest survey, which included the items on the pretest survey plus additional open- and close-ended items that elicited feedback regarding the feasibility, relevance, and appeal of all the VR prototype materials. The research staff reviewed the open-ended responses with the student and probed for additional feedback when needed, and the participant was asked to provide any additional feedback or final impressions. Finally, the participant was debriefed about the study, informed that they would receive the gift card incentive via email, given an additional opportunity to ask questions, and provided with contact information for research staff and university counseling services. Combined, these study activities lasted approximately 1 h.
Educator participants
To recruit educators, the research team directly contacted university faculty in academic departments (e.g., psychology, media studies) and staff members in student services (e.g., alcohol, drug, and violence prevention and response) via in-person requests and email. Educators who had routine contact with students were eligible to participate. After the educator participants completed the study consent form, research staff initiated the same procedures used with student participants to prepare and instruct the educator to engage with the VR modules. After the educator viewed the three VR module prototypes, research staff conducted an in-depth interview with each participant to obtain their feedback regarding the feasibility, relevance, and appeal of the VR modules. Each interview was approximately 10 to 15 min in duration.
VR modules
Development process
The VR modules presented provocative behavioral rehearsal scenarios that portrayed common challenges that college students face when living and studying with other students. The specific topics were identified in preliminary planning as highly relevant for college students after a review of the research literature. 29 The modules were filmed using VR180 video, a virtual reality video format that presents the user with a 180° visual panorama and 3D positional audio that are responsive to their movements in virtual space. A multidisciplinary panel of investigators and subject matter experts in alcohol and drug abuse, violence, prevention science, instructional design, and behavior change using VR and other new media technologies developed the prototype materials so that they were aligned with the goals of the LST program. The team worked to ensure that the content would be appealing and developmentally appropriate for the target age group. The panel developed a series of narrative branched scenarios based on a body of etiologic research on the risk and protective factors associated with adolescent and young adult problem behaviors. 28 Branched scenarios were designed to provide the learner with immersive virtual experiences in which they could select one of three options and observe the consequences of their decision. The panel then iteratively critiqued and finalized the scenarios with an emphasis on clarity, accuracy, innovation, appropriateness, engagement, and potential effectiveness.
Description of VR scenarios
The virtual settings, scenarios, decision points, and outcomes of the three VR module prototypes are described in Table 1. One of the prototypes was called “Good Party Gone Bad” and focused on bystander intervention skills practice. As shown in Figure 1, the scenario involves the learner sitting at a kitchen table at a dorm party. The learner observes Derek and Rebecca (names are pseudonyms) talking in an animated way (figure Frame A); then as Rebecca steps away Derek pours a powder into her drink (figure Frame B) and gestures to the learner not to say anything (figure Frame C). The learner then faces a decision point (figure Frame D) to either distract Derek and tell Rebecca what happened, act aggressively, or do nothing. If the learner selects the correct answer and uses appropriate bystander intervention skills, the result flashes forward where Rebecca is thanking the learner for watching out for her. If the learner selects an incorrect response, the negative outcomes appear as text and prompt the learner to try again. The other two prototype scenarios included “Copy Cat,” a branched scenario where the learner is prompted to use assertive communication skills to disrupt a situation in which one student plagiarizes the homework of another student, and “Dirty Dishes,” a branched scenario where the learner is prompted to use conflict resolution skills to deescalate an argument between two roommates. In each scenario, on-screen actors speak directly to the learner and ask for their input and encourage their full engagement in the interactions. Screenshots from the “Good Party Gone Bad” VR module. In this scenario, the learner is sitting at a kitchen table at a dorm party and observes Derek* and Rebecca* talking about how they’re excited that midterms are over (Frame A). Rebecca sets her drink down and says that she’ll be right back, and while she is away Derek pours a powder into her drink (Frame B). Derek makes eye contact with the learner and makes a gesture to not say anything (Frame C). The learner is prompted to make a decision: distract Derek and have someone tell Rebecca what happened, grab the drink and throw it on him and yell “what are you doing?”, or do nothing and walk away (Frame D). If the learner selects the correct answer and uses appropriate bystander intervention skills to address the situation, the result flashes forward to the next day where Rebecca is thanking the learner for watching out for her. If the learner selects one of the incorrect answers, the negative outcomes appear as text and prompt the learner to try again. (*Pseudonyms. At the request of the editor, black bars have been placed over the eyes of the actors in the screenshots above; the black bars were not in the original VR modules.)
Measures
A pretest and posttest survey assessed life skills knowledge and anti-violence attitudes. Life skills knowledge was assessed using 12 true/false survey items developed by the research team to assess the specific content taught in the college program and used in previous research. 20 Four items measured assertiveness knowledge (e.g., “Assertive behavior involves acting in your own best interest.” [true]); four items measured conflict resolution knowledge (e.g., “Healthy relationships should involve no conflicts.” [false]); and four items measured bystander intervention knowledge (e.g., “Being an active bystander means you should risk your personal safety to help another person i.e. in danger.” [false]). An overall skills knowledge score was calculated based on the total number of items answered correctly. The remaining pre-post survey items assessed anti-violence attitudes among participants, using items from the Beliefs about Aggression and Alternatives scale 30 (α = 0.72; e.g., “There are better ways to solve problems than fighting.”), and Attitudes Toward Interpersonal Violence 31 (α = 0.75; e.g., “The best way to stop a fight before it starts is to stop the argument [problem] that caused it.”). Response options were on a 4-point Likert scale from “strongly disagree” to “strongly agree.” An overall anti-violence attitudes score was calculated based on the sum of the items, with higher scores representing greater agreement.
After completing the full set of three VR modules, participants were asked a series of close-ended questions assessing whether they disagreed or agreed (strongly, somewhat) that the set of VR modules were easy to use, innovative, and engaging. They were also asked whether the conflicts presented in the VR modules felt realistic and whether they could see the VR modules being implemented on a college campus. Next, participants were asked whether the modules would be appropriate, relevant, appealing, and feasible for first-year students. Scores to summarize participant responses were obtained by calculating the percentage of participants that agreed with the statements. Finally, student participants were asked a set of open-ended survey questions regarding the VR modules that focused on overall feasibility (“How might the VR modules best be implemented at your college?”) and appeal (“What parts of the VR modules do you think students would find most/least appealing?”). They were then probed to provide further information on their written responses, and asked to make recommendations concerning how the VR modules might be further refined and improved.
Educators were interviewed after viewing the set of three VR modules. They were asked to provide their overall impressions on what they thought students would find most/least appealing about the VR modules, how the VR modules might best be implemented at their college, and what recommendations they have for improving the VR modules.
Data analysis plan
A multiple methods strategy was used to analyze the student and educator feedback. Paired t-tests were used to examine pretest to posttest changes in life skills knowledge and anti-violence attitudes among student participants. To analyze the qualitative data, the research team identified major themes by systematically reviewing responses to the open-ended survey and interview questions. Using standard techniques for qualitative data, 32 themes were generated, adapted, reviewed, refined, and aggregated in an iterative process, until independent and parsimonious themes and exemplar statements were identified. The goal of the qualitative analyses was to identify a set of central themes raised by participants regarding their impressions of the VR module prototype materials.
Results
Quantitative and qualitative findings from students are presented below, including: (1) pretest to posttest changes in life skills knowledge and anti-violence attitudes; (2) student overall impressions of the VR modules; and (3) student general perceptions about the fit of the VR modules for first-year students. Feedback collected from college educators regarding the VR module prototype materials is also summarized.
Skills knowledge and attitudes
Pretest and posttest means for life skills knowledge and anti-violence attitudes.
Student feedback: overall impressions
As shown in Figure 2, the vast majority of students agreed (strongly/somewhat) that the modules were easy to understand (100%), easy to use (97.5%), innovative (100%), and engaging (92.5%). Most students (95%) agreed that the conflicts presented in the VR modules felt realistic. All students (100%) agreed that the VR modules could be implemented on a college campus. This level of agreement suggests that the VR modules were generally very well received by students. Students also provided useful qualitative feedback regarding various aspects of their experience and perceptions including how the modules could be improved. Below we summarize student feedback regarding their overall impressions into four categories including: (1) usability; (2) innovation; (3) engagement/realism; and (4) potential for implementation on college campuses. Student reactions to virtual reality module prototypes.
Usability
As shown in Figure 2, all participants (100%) agreed that the instructions were understandable and most (97.5%) said that the modules were easy to use. A small number of students disagreed (2.5%) or agreed somewhat (7.5%) that the VR modules were easy to use. For students with no experience with VR, it took some practice to become familiar with using the VR headset and handheld controllers to view, interact with, and navigate through the immersive world. A few students suggested it would be helpful to have a more extensive tutorial available before interacting with the VR modules to facilitate ease of use.
Innovation
All students (100%) agreed that the modules were innovative. They described the VR modules as new, exciting, and fun and that they allowed for a different kind of learning experience. One student reported that “The VR experience put me more directly into the situation and allowed me to connect on another level. It was like watching a film and playing a videogame at the same time.” Another student commented that “Being in the scenario where tensions were high was challenging, but I knew I was in a VR headset which helped me stay calm. It was a really cool experience.” A small (15%) portion of the study sample agreed somewhat that the content was innovative. Students suggested that the modules could be made more innovative by including more interactive features such as the ability to speak their responses and have their speech recognized within the module, and expanding their ability to view, explore, and interact with the virtual setting in full 360-degree video.
Engagement/realism
Students were asked to provide feedback on how engaging and realistic the VR modules were. Findings indicated that 92.5% of students agreed that the modules were engaging and most (95%) agreed that the conflicts presented felt realistic. Students commented that the experience of being “present” in real life situations and participating in the outcome of the scenarios was highly engaging. One student reported that the dramatic situations shown in the branched scenarios made them feel like they were “a part of the situation in real time.” They commented that the VR scenarios showed relatable life-like situations and the on-screen characters looked and interacted with each other in realistic ways. Several students commented that having the virtual actors look at them and ask them for input was very engaging. One student said “The modules demonstrate how it actually feels in the situations, and the eye contact with the characters makes it feel like you are in the room.” Another said “The immersive nature of the experience makes it interesting and easy to pay attention to what is happening.” Others noted that the VR scenarios were “hyper-realistic” and “as real as it gets.”
Some students were less favorable regarding how engaging and realistic the VR content was. Some (7.5%) students disagreed and 27.5% agreed somewhat that the modules were engaging, and 5% of students disagreed and 25% agreed somewhat that the conflicts felt realistic. Some students commented that the tense situations portrayed in the VR modules de-escalated too easily and would play out in a more complex way in real life. Some mentioned that having more than three response options at each decision point would be preferable, and that the correct response was sometimes too obvious. Students suggested several ways that the modules could be made more engaging and realistic. Students suggested making the scenarios longer and showing the outcome of all responses in video, rather than showing on-screen text for wrong answers. They also suggested that the modules could be more realistic by increasing the complexity of the scenarios, adding more decision points within each module, including a greater number of response options for each decision point, and including having more than one correct option to see how different appropriate responses would play out in terms of the consequences.
Potential for college implementation
All students (100%) agreed that the modules could be implemented on college campuses. Specifically, students suggested that the VR modules could be incorporated into mandatory training at freshman orientation or as part of a first-year seminar, as an in-class activity in a relevant college course, at violence prevention events on campus, or as part of an exhibit in the library or student union offered with free snacks. They also commented they could see VR modules being implemented in a group setting, where participants would have the opportunity to discuss the scenarios after viewing them. The general feedback indicated that students believed that VR modules could be implemented in a variety of ways as part of the college experience. Indeed, most students said they would prefer the VR materials to what their university currently offers for drug and violence prevention because the VR modules were more realistic and engaging, with one student commenting that the modules “offer a wider range of scenarios that give more of an educational experience than what is typically provided.”
Student feedback: fit for first-year students
Students were asked how they felt first-year college students in general would react to the VR modules. Below we summarize student feedback regarding (1) appropriateness/relevance; and (2) appeal.
Appropriateness/relevance
All students (100%) agreed that the modules were appropriate and relevant for first-year students. Students described the scenarios presented as common experiences among college students, displaying conflicts that are relatable and highly applicable to college life. Several students noted that the conflicts mirrored problem situations that they have encountered at college. One student expressed that “Dishes are a common theme that cause roommate conflict. Being a resident hall assistant, I deal with this often.” Another commented “I think everyone who lives at college eventually has to navigate a roommate conflict.” Others commented that being involved in academic cheating happens frequently and was relatable, realistic, and relevant. One student noted “I’ve experienced cheating situations in real life many times this semester and this scenario hit home.” Overall, students indicated that the content on how to say no and stand up for oneself was especially relevant for first-year students. A student commented that “The VR scenarios were helpful in learning how to stand up for yourself and others in different difficult situations.” A significant portion of students, 1 in 4, agreed somewhat that the modules were relevant for first-year students. One student said the cheating situation was not realistic because some students wouldn’t care if someone cheated off them. Another said that “No one would try to slip something into a drink when someone is watching and then ask them to stay quiet.” Students suggested that the VR branched scenarios could be improved by adding a greater variety of situations that better reflect the many different types of conflicts that college students face in their daily lives.
Appeal
Most participants (97.5%) agreed that the modules would be appealing for first-year students. Participants reported that the immersive and realistic environment of the VR modules was interesting and fun and enjoyed the ability to move around and participate in life-like situations and then watch as they come to a resolution. They reported that the VR modules encouraged them to think about their own decisions and consider realistically what they would do when faced with conflict. Students commented that, “It was great to do this in VR, to allow students to walk through a possible scenario before having to deal with it in real life” and “Learning new techniques to intervene in risky situations was a great experience and very useful.” A small percentage of students disagreed (2.5%) that the VR content would be appealing for first-year students and a larger portion (32.5%) agreed somewhat that the modules would be appealing. To improve appeal, students recommended longer and more nuanced scenarios and story lines that involved more complex decision making, more actors in the scenarios, more time interacting with the on-screen characters prior to the conflict being presented, and adding gamification elements to the VR modules such as quests, competitions, rewards, and more personalization and customization for each learner.
Educator feedback
Findings from the educators found universal agreement that the program was feasible, relevant, useful, and appealing for violence prevention and that the VR modules were highly engaging. Educators described the modules as attractive, realistic, and helpful tools that allowed for an interactive and immersive experience. Educators reported that the modules could promote discussions among students, and possibly better engage students who would be too shy to participate in role play scenarios in a classroom setting. They indicated that they could envision the modules being implemented on their college campus for new student orientation, as part of dormitory life or residential hall assistant training, and to supplement or replace existing prevention programming. Educators described that they thought students would find the immersive and interactive experience of VR most appealing. When asked about improving the current modules, educators suggested adding more decision points and response options to choose from for each scenario would make the experience more nuanced and realistic. They recommended that the storyline and video should continue after the user selects a wrong answer. When asked about what parts of the VR modules they thought students might find most difficult, educators reported that some students may have a hard time choosing the correct response, and that some students may experience discomfort or disorientation if they have never used a VR headset before. Educators also expressed that from an accessibility standpoint, the VR technology may not be viable for all students, such as students who are vision impaired.
Discussion
The current study developed and pilot tested VR technology as a tool for skills practice as part of an effective resilience-based substance abuse and violence prevention program for college students. After viewing e-learning modules with prevention content related to building healthy relationships, assertive communication, and bystander intervention, students practiced cognitive behavioral skills in an immersive VR180 environment. Branched scenarios provided opportunities for students to practice assertive communication, negotiation, compromise, conflict resolution, and bystander intervention skills for preventing risk behaviors. Findings showed increases in life skills knowledge and anti-violence attitudes among students from the pretest to posttest assessments. Students and educators were generally enthusiastic about the VR prototypes, rating them as feasible, relevant, appealing, engaging, and innovative for prevention. Participants provided feedback on ways to improve the VR experience. The primary themes mentioned had to do with including a greater variety of conflict situations, providing more branched scenarios within each module, having a more nuanced set of response options to select from in each branched scenario, and including a more complete presentation of all scenario outcomes.
Supplementing evidence-based prevention with digital technologies
Recent research has found that digital technologies offer considerable potential for preventing substance abuse and violence, particularly when used to supplement traditional evidence-based classroom prevention. For example, a hybrid version of the LST middle school program that included a series of e-learning modules plus a reduced set of classroom sessions was found to effectively prevent multiple forms of substance use among students. 19 Further, educational video game modules that supplemented LST classroom sessions were effective in preventing bullying and cyberbullying perpetration among middle school students. 18 The current study is a continuation of this body of research focusing on the efficacy of digital intervention materials as supplements to the evidence-based LST prevention model. Findings from the present study demonstrated that VR modules used to supplement e-learning content were viewed by college students and educators as feasible, relevant, appealing, engaging, and innovative. Taken together, these studies suggest that digital technologies can engage youth and hold significant promise as supplements to traditional prevention approaches. By reducing burdens on classroom time and teachers, supplementary digital content may increase the reach of prevention programming, make implementation more feasible, improve implementation fidelity, and maximize cost-effectiveness in real-world settings.
Virtual reality and skills training for youth
In the present study, VR module prototypes were developed and evaluated as a virtual practice space for college students as a supplement to evidence-based e-learning modules. The materials were designed to address shared risk and protective factors for substance use and interpersonal violence for this population. This study builds upon several recent studies that have developed and evaluated immersive VR environments as tools for skills building for health promotion and risk reduction among youth. A small but growing number of studies have tested VR scenarios in the context of substance use and violence prevention among youth. A systematic review of VR-based alcohol prevention research in adolescents 33 identified four studies that exposed youth to immersive environments with alcohol-related scenarios (e.g., peer pressure to drink in a party situation, negative alcohol consequences). Most of the studies were qualitative in nature, and adolescents responded well to the realistic VR environments, providing positive feedback about the novelty and realism of the immersive experience. They reported high levels of satisfaction with the degree of interactivity in the VR simulation, such as the ability to control the environment and choose one’s path through the scenarios. Across studies, some positive effects were observed on outcome measures, including changes in alcohol knowledge, attitudes, and intentions, particularly among students who reported the highest levels of satisfaction with the VR technology.
In a study of middle school youth, 34 a VR-based vaping prevention program was tested that included content on the health risks of vaping and nicotine use along with a virtual environment to practice refusing peer pressure to vape. This study included a randomized design, with classrooms assigned to the VR condition or a control condition that received treatment as usual. Findings indicated that students who received the VR intervention provided positive ratings and high levels of satisfaction with the experience, gameplay, and game-based decisions. They also reported improvements in knowledge related to e-cigarettes and addiction and greater perceptions of harm compared to a control group. However, no differences between the intervention and control groups were observed in terms of e-cigarette use. The lack of behavioral effects may have been due to the relatively short follow-up of 6 months, and the study took place during lockdowns for COVID-19, which may have reduced access to e-cigarettes and opportunities to engage in vaping with friends.
Studies have investigated the use of VR as a violence prevention tool, focusing on using immersive environments to provide practice opportunities for bystander intervention. A study of middle school students tested VR scenarios as a supplement to classroom sessions on bullying prevention. 35 In the VR scenarios, students were placed as virtual witnesses to altercations in social situations that took place in a school hallway or party scene, and the modules emphasized how to be an active bystander and stand up for victims. The scenarios focused on ways to make a difference with small and realistic actions for intervention and portrayed the consequences of common ineffective responses to bullying. Findings indicated that empathy, school belonging, and willingness to intervene increased in the intervention group relative to controls, and positive behavior changes were observed for traditional bullying but not for cyberbullying or relational aggression.
The present study adds to a growing body of literature showing that VR technology is a promising approach for skills training activities that address youth health risk behaviors. Given the increasing availability, decreasing costs, and future scalability of VR technology, using VR to supplement evidence-based prevention represents an innovative opportunity to address health risk behaviors using digital technologies that are popular among youth. VR systems, such as the Meta Oculus Quest 2 system used in the present study, are operated as stand-alone devices with no need for external equipment, and have become increasingly affordable. The positive findings suggest that the development of VR modules has the potential to strengthen existing effective prevention programs for college substance misuse and violence and may maximize participant engagement and learning while providing the ability to practice and refine new skills in a safe virtual environment. The present findings also provide insight into ways to make VR more engaging and appealing to college students by creating branched scenarios, decision points, and pathways through the experience that that are more detailed, nuanced, and realistic, and by presenting a fully realized experience that displays all scenario outcomes.
Strengths and limitations
There are several conceptual and methodological strengths of the present research. The study had several innovative features in that it focused on prevention within a broader positive youth development framework, was adapted from the Life Skills Training prevention model which has been supported by an extensive body of rigorous research, and combined state-of-the-art VR technology with evidence-based e-learning modules. The skills training focus of the branched scenarios was based on an established body of etiological research on shared risk and protective factors for substance use and interpersonal violence (e.g., deficits in self-regulation and social competence). The VR modules provided realistic but safe environments for cognitive behavioral skills practice in immersive, developmentally-appropriate scenarios that are common among young adults in college settings.
There are several limitations that warrant consideration. First, although significant improvements were observed in life skills knowledge and anti-violence attitudes from the pretest to posttest, it was not possible to isolate this effect entirely to the VR modules. Second, the knowledge and attitudes measures were brief in this pilot and feasibility study and future research on this intervention approach should use more comprehensive measures in a randomized controlled trial. Third, the study was limited by a short-term evaluation and a small sample size. It is possible that feasibility and appeal differed by gender or other participant characteristics, but these were not analyzed given the small sample size. Finally, the cost of producing VR content as well as that of high-quality headsets may be perceived by some educators and administrators as prohibitive, although costs are likely to continue to decrease over time.
Conclusions
The present study found that VR modules for skills practice, as a supplement to e-learning substance use and violence prevention content, were perceived as innovative, relevant, feasible, and appealing to the college students and educators included in the study. VR technology appears to have considerable potential as an engaging, effective, and cost-effective component of skills-based preventive programming for college students. If effective, VR modules combined with evidence-based prevention materials could be widely disseminated as an innovative and engaging primary prevention approach to reduce substance use and violence among students at 4-year colleges, community colleges, universities, graduate/professional schools, trade schools, and career and technical schools. Additional research is needed to test whether and how the skills practiced in VR scenarios transfer to students’ real-life decision-making and health risk behaviors and the duration of these effects. Studies should examine how VR scenarios can address the specific needs and backgrounds of individual students, including those from diverse backgrounds. For example, VR technology could allow for scenarios and simulation pathways to be tailored to individual student risk profiles, gender, and other social characteristics. Further study is needed to better understand how to present risky situations in VR scenarios in ways that are engaging and realistic but not triggering to those with previous victimization experiences. It will also be important to identify ways to effectively make VR technologies more accessible to students including those at smaller or less well-funded universities, as well as how to widely disseminate and integrate VR prevention training into existing university prevention activities.
VR for reducing adolescent risk behaviors is an emerging area of research, focusing mostly on developing VR modules that are appealing and feasible. Additional rigorously designed randomized controlled trials with long term follow-up will be important for examining whether VR technologies can produce behavioral outcomes and the duration of these effects. Such studies can help determine whether VR provides sufficient practice experience as a supplement to e-learning modules. If so, this would be important and suggest that prevention programming could be delivered with limited college staff facilitation, increasing its feasibility, fidelity, and reach. In summary, while a growing body of studies show that VR may be a very useful tool for reducing youth health risk behaviors, more rigorous controlled trials are needed.
Footnotes
Acknowledgements
The authors would like to acknowledge Eric Kondel for his assistance with various phases of the study, and Anahita Asghari-Kamrani for her assistance in reviewing the literature and in coding the qualitative data.
Author contributions
K.W.G. and C.W. supervised all study activities, including study design, data collection, interpretation of results, and co-wrote the first draft of the manuscript. S.M.S. was involved in data collection and contributed to the writing of the manuscript. G.J.B. provided oversight of this study and intervention development, and contributed significantly to the writing of the manuscript. All authors have read and agreed to the published version of the manuscript.
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Authors G.J.B., C.W., and S.M.S. are employees at National Health Promotion Associates (NHPA), which markets the Life Skills Training prevention program. K.W.G is a former employee of NHPA and currently serves as a consultant to NHPA.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, R43 CE003173.
Ethical statement
Data availability statement
Data available upon reasonable request.