Toward Adaptive Virtual Reality Systems: Understanding Emotional and Physiological Responses in VR Game

Abstract

Virtual reality (VR) affective computing is expanding to include systems that can identify and respond to human emotions. This study explores emotional responses in VR using the game Pong to monitor these reactions under different gameplay conditions. It investigates how game pace and opponent behavior affect emotions, assessed through physiological metrics like electrocardiogram, galvanic skin response, electromyogram, and player behaviors. Results from three game variations—slow-paced, fast-paced, and lag-induced—showed distinct emotional and physiological responses. Significant differences were found in arousal and valence levels, with varying heart rate and skin conductance among the game conditions. These findings suggest that VR game design can significantly influence emotional states, providing insights that could help develop more engaging and responsive VR systems. Thus, this study demonstrates the potential of VR to evoke and measure specific emotional responses, contributing to the fields of human-computer interaction, and affective computing.

Within affective computing in virtual reality (VR), researchers are developing systems that can better understand and respond to human emotions (Desselle et al., 2020; Marín-Morales et al., 2020; Picard, 2000). They leverage models such as the circumplex model of affect, which defines emotions based on dimensions of arousal, valence, and dominance (Russell, 1980). With its ability to create realistic and immersive environments, VR technology offers a significant advantage for affective computing by allowing the analysis of physiological responses—including heart rate via electrocardiogram (ECG), skin conductance through galvanic skin response (GSR), and muscle activity measured by electromyogram (EMG)—and adapting environments in real-time to optimize user interaction (Bayro et al., 2022; Bayro et al., 2023; Järvelä et al., 2020). This study adopts an innovative approach by utilizing a VR Pong game, previously used as both a rehabilitation tool and a game (Darzi & Novak, 2017). The game has been modified through Unity’s Air Hockey Game Kit (Air Hockey Game Kit: Systems, n.d.) to create a controlled environment for examining how changes in game dynamics impact a player’s emotional state. The findings provide valuable insights for the development of emotionally responsive systems.

We recruited 30 participants with moderate familiarity with VR and between the ages of 18 and 62. Three versions of the VE were created to evoke different emotions: slow-paced, fast-paced, and lag-induced. Participants were equipped with various sensors, including ECG, GSR, and EMG, to monitor physiological responses while playing VR Pong. We used a counterbalanced design for each session to minimize order effects. The Self-Assessment Manikin (SAM) was used to quantitatively measure arousal and valence after each scenario (Bradley & Lang, 1994).

Statistical analysis revealed significant differences in emotional responses across the three game conditions. Namely, the SAM analysis indicated variations in arousal and valence levels, with the fast-paced game evoking positive arousal and valence, the slow-paced game eliciting positive arousal but negative valence, and the lag-induced game resulting in negative arousal and valence.

Furthermore, analysis of physiological signals, including ECG, GSR, and EMG, highlighted significant differences between the game conditions. For instance, features derived from ECG signals showed significant power and heart rate differences between game variations. GSR signals exhibited variations in the number of nonspecific skin conductance responses. EMG activity analysis revealed significant differences in features related to muscle activity, such as standard deviation and mean frequency, across the game conditions. These findings contribute to the existing body of knowledge by demonstrating the influence of VR game dynamics on emotional responses and physiological reactions. Understanding how different game characteristics affect emotional states and physiological arousal can inform the design of VR experiences tailored to evoke specific emotional responses and enhance user engagement, contributing to advancing human factors and ergonomics in VR technology.