Abstract
Objective:
Applied games are considered a promising approach to deliver mental health interventions. Nonspecific factors such as expectations and motivation may be crucial to optimize effectiveness yet have not been examined so far. The current study examined the effect of expectations for improvement on (1) experienced fun and positive affect, and (2) in-game play behaviors while playing MindLight, an applied game shown to reduce anxiety. The secondary aim was to examine the moderating role of symptom severity and motivation to change.
Materials and Methods:
Fifty-seven participants (47 females; 17–21 years old) preselected on anxiety symptoms viewed a trailer in which MindLight was promoted as either a mental health or an entertainment game. These trailers were used to induce different expectations in participants. Participants subsequently played the game for 60 minutes. Before playing, participants filled out questionnaires about their general anxiety symptoms, motivation to change, state anxiety, affect, and arousal. While playing, in-game behaviors and galvanic skin response (GSR) were recorded continuously. After playing, state anxiety, affect, and arousal were measured again as well as experienced fun.
Results:
Participants in both trailer conditions showed increases in state anxiety, arousal, and GSR. Expectations did not influence experienced fun and positive affect, nor in-game behaviors. In addition, no moderation effects of motivation to change and symptom severity were found.
Conclusion:
Experiences and engagement with MindLight were not influenced by expectations, motivation to change, and symptom severity. For future research, it is recommended to examine individual differences in these effects, and long-term and more distal outcomes and processes.
Introduction
There has been an increasing interest in the use of applied games to treat and prevent mental health problems.1,2 Due to their intrinsically motivating features and their high accessibility and potential for scalability, applied games are considered a promising and cost-effective approach to improve access to mental health care.1,3 The primary focus in the development of applied games has been on translating evidence-based
Presumably, the most relevant nonspecific factor to examine in applied games is individuals' expectations for improvement.21,22 Previous research has shown that expectations drive a large majority of intervention effects, 13 in particular, in experimental game design studies. 23 Although commercial videogames are usually promoted for their entertainment value, applied games are often introduced with an explicit (mental) health aim, which naturally induces expectations for improvement. It is unknown how expectations relate to players' experiences of a game and their engagement with it. Therefore, the primary aims of the current study were to examine the effect of expectations for improvement on (1) experienced fun and affect, and (2) in-game play behaviors while playing an applied game for mental health.
The effect of expectations on players' experiences and in-game engagement may be moderated by two additional nonspecific factors, namely motivation to change (i.e., individuals' willingness to change symptoms or problems they are experiencing) 24 and symptom severity. An individual more motivated to change mental health symptoms and/or experiencing more (severe) symptoms may find an applied game with an explicit mental health aim more personally relevant, possibly leading to higher engagement with the therapeutic techniques and a more positive experience of the game (i.e., experienced fun and affect).15,25,26 On the other hand, however, it might be that individuals with more (severe) symptoms engage less with the therapeutic techniques as they may fear unwanted confrontation with their mental health problems 27 (e.g., a game aimed at emotion and stress management may imply confrontations with negative emotions and stress for some individuals). The secondary aim of the current study was to examine the moderating role of motivation to change and symptom severity.
Design of the present study
In the current study, we used the applied game MindLight, designed to reduce anxiety symptoms among youth.28,29 Previous research has compared MindLight with a CBT-based indicated prevention program in children (7–12 years old) 30 and to online CBT-based psychoeducation in adolescents (8–16 years old), 31 finding evidence for its overall effectiveness. Previous research also suggests that both specific and nonspecific factors (expectations and motivation) play a role in MindLight.32,33
Expectations for MindLight were experimentally manipulated by showing participants a teaser trailer, in which MindLight was promoted as a mental health game or as a regular entertainment game.23,34 The primary outcomes were experienced fun, positive affect, and in-game play behaviors. Because MindLight has been specifically designed to induce anxiety to train youth to regulate this anxiety,30,33 we also examined changes in (self-reported) state anxiety and arousal. The study design, hypotheses, and analyses were preregistered on the Open Science Framework (OSF; https://osf.io/6gmwv) 35 and deviations from the planned methodology are uploaded on OSF (https://osf.io/j7mvu). Exploratory analyses were performed on changes in galvanic skin response (GSR; i.e., the small changes in the amount of moisture or perspiration on the surface of the skin), to have an objective indicator of arousal complementing the self-reported measures.
Methods
Participants
Participants were 57 psychology students, who were between 17 and 21 years old, primarily Caucasian, and indicated to be moderately experienced with playing videogames (see Table 1 for descriptives). All participants were preselected on elevated levels of anxiety (see preregistration). The Queen's University Health Sciences & Affiliated Teaching Hospitals Research Ethics Board (HSREB) granted ethics approval for the current study (code number: 6019310 PSYC-187-16).
Means, Frequencies, and Standard Deviations or Percentages of the Demographic and Study Variables for the Total Sample and for Each Experimental Condition
Bootstrapping the independent
For the
The χ 2 -test included males and females (one nonbinary participant was removed); 2 cells (50%) had an expected count <5 (minimum expected count was 4.50).
The χ 2 -test included the categories “white” versus “other.”
On a 10-point scale ranging from “0 = not at all experienced” to “10 = expert.”
One value was winsorized to ±3.5 SD from the mean.
Levene's test was significant and equal variances could not be assumed.
Procedure
Participants signed informed consent and filled out a questionnaire measuring demographics, anxiety symptoms, and motivation to change. After that, participants viewed a neutral video to measure their baseline GSR 36 and completed a questionnaire measuring their state anxiety, affect, and arousal. Next, participants viewed a mental health or an entertainment trailer and played MindLight for 60 minutes on a 15.6-inch laptop. After having played the game, participants again filled out questions about their state anxiety, affect, and arousal, as well as questions about their experiences with the game and questions related to the manipulation checks.
MindLight and experimental manipulation
MindLight is a three-dimensional neurofeedback game designed to reduce anxiety symptoms among youth.28,29 In the game, Little Arty (the player) needs to save his grandma who succumbed to evil forces. He finds a magical hat that teaches him (and the player) how to use his “mindlight”, a beam of light coming from the antenna attached to the magical hat. The “mindlight” is controlled through the one-channel dry-sensor electroencephalogram (EEG) headset that the player wears, 37 and which responds to the real-time relaxation of the player (neurofeedback training 38 ): when the player becomes more relaxed, the light becomes brighter providing more light in the game environment, and making it possible to chase away or uncover “fear events” (exposure training 39 ) and effectively engage with the (attention bias modification40,41) puzzles and other objects (e.g., unlock hiding spaces and turn on ceiling lights, which both prevent that fear events will attack the player). For more information, see previous studies on MindLight.30,32,33,42,43
Half of the participants (
Materials
Experienced fun
Participants answered “How much did you like playing MindLight?” on a 10-point scale.
Affect and arousal
Participants indicated on two manikin-based scales (ranging from 1 to 5) how they felt at that moment (self-assessment manikin [SAM]). 44 Each manikin is a graphical depiction of various points along the affect/arousal dimension. For affect, the SAM ranged from an unhappy and frowning manikin (1) to a happy and smiling manikin (5). For arousal, the SAM ranged from a sleepy and relaxed manikin (1) to a wide-eyed and excited manikin (5).
In-game play behaviors
While playing MindLight, on-screen output was recorded using the Open Broadcast Software (https://obsproject.com). In-game play behaviors were coded in Noldus Information Technology
45
following an adapted version of the
State anxiety
The state scale of the State-Trait Anxiety Inventory46,47 consists of 20 items (αpre = 0.90 and αpost = 0.92) and asks participants to indicate how they feel at this moment (e.g., “I am tense”; on a 4-point scale).
Galvanic skin response
During the baseline task (i.e., viewing a neutral video) and while playing MindLight, GSR was recorded continuously with Biopac AcqKnowledge 4.2 software 48 and MP150 amplifier. 36 GSR data files were trimmed to 120 seconds and 3600 seconds for the baseline task and gameplay, respectively, and cleaned and processed using the AcqKnowledge software. No smoothing was applied to the data, but a low-pass filter was used to improve the signal quality of the entire waveform (fixed frequency was set at 0.5 Hz). An overall GSR mean value for baseline and gameplay was calculated, as well as GSR mean values for six 10-minute timebins (i.e., dividing the 60 minutes of gameplay into timebins of 10 minutes). Then, the GSR percent change from baseline was calculated for each participant to control for individual differences and to facilitate interpretation across participants49,50 (for more details see OSF; https://osf.io/j7mvu).
Motivation to change
The contemplation subscale of the University of Rhode Island Change Assessment questionnaire
51
consists of eight items (α = 0.73), measuring the extent to which participants are aware of their “problems” (as identified with the Beck Anxiety Inventory [BAI],
Symptom severity
The BAI52,53 measures various symptoms of anxiety with 21 items (α = 0.94). Participants indicated the degree to which they were bothered by each symptom on a 4-point scale.
Trailer manipulation check
To examine whether the two trailers induced different expectations, participants answered two open questions: “What were your impressions of MindLight right after you watched the trailer?” and “What did you expect from MindLight based on the trailer (before you played the game)?” In addition, participants answered the following
Statistical analyses
First, trailer manipulation and randomization were checked. Bootstrapped Pearson correlations between study variables are reported in Table 2. Bootstrapped paired t-tests were used to examine changes in affect, state anxiety, and arousal. The remaining preregistered research questions were examined within a (hierarchical) regression framework (controlled for high correlations; see preregistration).
54
Univariate outliers were winsorized to ±3.5 standard deviation from the mean before conducting the analyses. Because some study variables were not normally distributed, all regression models were bootstrapped with
Pearson Correlations between the Study Variables
All correlations were bootstrapped with
One value was winsorized to ±3.5 SD from the mean.
CI, confidence interval; GSR, galvanic skin response.
For GSR, univariate outliers were winsorized to ±3.5 standard deviation from the mean, both for the overall GSR percent change value and the timebin values. The exploratory analyses for GSR included (1) a bootstrapped one-sample
Results
General manipulation, trailer manipulation, and randomization check
Participants who were aware of the study aims (
Trailer Manipulation Check
Experienced fun and change on affect
Experienced fun did not significantly differ between the two trailer conditions. The interactions between trailer condition and motivation to change or anxiety symptoms did not have a significant effect on experienced fun (Table 4). For positive affect, we found a significant decrease from pre- to posttest [
Hierarchical Linear Regression Analyses Predicting Experienced Fun and Affect at Posttest
Trailer condition was coded as 0 = entertainment trailer, 1 = mental health trailer.
CI, confidence interval; SE, standard error.
In-game play behaviors
Trailer condition did not significantly predict any of the in-game play behaviors (Tables 5 and 6). The interaction between trailer condition and motivation to change did not have a significant effect on any of the in-game play behaviors, with the exception of fear attempt (Table 5). A significant positive effect of motivation to change on fear attempts was found in the MH condition,
Hierarchical Linear Regression Analyses Predicting the Engaged In-Game Play Behaviors
Trailer condition was coded as 0 = entertainment trailer, 1 = mental health trailer.
CI, confidence interval; SE, standard error.
Hierarchical Linear Regression Analyses Predicting the Avoidant/Safety In-Game Play Behaviors
Trailer-condition was coded as 0 = entertainment trailer, 1 = mental health trailer.
CI, confidence interval; SE, standard error.
Change in state anxiety and arousal
Participants in both trailer conditions reported increased state anxiety [
Hierarchical Linear Regression Analyses Predicting Arousal and State Anxiety at Posttest
Trailer-condition was coded as 0 = entertainment trailer, 1 = mental health trailer.
CI, confidence interval; SE, standard error.
Exploratory analyses
Overall GSR percent change during gameplay was significantly higher than zero with a mean difference of 32.92 [
Hierarchical Linear Regression Analyses Predicting overall GSR Percent Change
Trailer condition was coded as 0 = entertainment trailer, 1 = mental health trailer.
CI, confidence interval; SE, standard error; GSR, galvanic skin response.
For the Repeated Measures ANOVA, Mauchly's Test of Sphericity indicated that the assumption of sphericity had been violated, χ
2
(14) = 117.92,
Mean Galvanic Skin Response Percent Change for the Six Timebins for the Total Sample and for Each Experimental Condition
Bootstrapped with
Bootstrapping the independent t-tests with
Levene's test was significant and equal variances could not be assumed.
Within-subjects polynomial contrasts, however, showed a significant quadratic trend for GSR [
Paired Samples
Bootstrapping the paired samples
Discussion
The current study examined the effect of participants' expectations for improvement (i.e., playing a mental health game or a regular entertainment game) on the following outcomes: (1) experienced fun and positive affect, and (2) in-game play behaviors while playing MindLight, an applied game shown to reduce anxiety symptoms in several randomized controlled trial (RCT) studies.30,31,33,43 We also investigated changes in state anxiety, arousal, and GSR. The secondary aim was to test the moderating role of motivation to change and symptom severity.
Our findings that expectations did not influence experienced fun and affect, are in line with previous research showing that young adults experiencing mental health symptoms found a commercial videogame promoted as a mental health game similarly attractive and fun as the same game being promoted for its entertainment value. 55 More importantly, players' game experiences and affect were not influenced by the mental health messaging. 55 We also found that expectations did not predict in-game behaviors and that participants in both trailer conditions showed similar increases in state anxiety, arousal, and GSR. Although players can explore and progress through the game in a variety of ways, MindLight's design seems to ensure that players engage similarly with the game, regardless of their expectations about the game. Because engagement with the therapeutic techniques is necessary to be successful at the game, players who are unaware of the mental health aim still end up playing the game in a similar way as players who are aware of the mental health aim and may also benefit from it. Similarly, a previous study showed that initial anxiety levels were not associated with in-game play behaviors. 32 Given the current findings, game designers may want to design applied games in such a way that players are encouraged to engage with the therapeutic techniques, regardless of their expectations about the game.
Regarding the secondary aim, we found no moderation of motivation to change and symptom severity, with the exception of one significant interaction between expectations and motivation to change on fear attempts. Given the small sample and multiple interactions that were tested, it could well be a chance finding and hence will not be further elaborated on. It might be that no moderation effects were found because individual differences have cancelled out some of the effects. For example, not all participants with equal levels of anxiety may have perceived the mental health message as personally relevant.25,26,35 In addition, expectations may not only be affected by an explicit mental health aim but may depend on other personal characteristics, such as gender, age, race, dispositional optimism, personality, treatment history, and beliefs about and experiences with applied games.20,26,35,56 Future research may want to examine the role of perceived personal relevance26,57 in combination with other individual differences. Future research may also investigate the role of nonspecific factors on the long-term as well as more ecological valid contexts, such as voluntary choice for, prolonged engagement with, and ongoing use of an applied game for mental health.
A limitation of the study is the modest sample size, only allowing detection of medium-sized effects. Second, participants were preselected on elevated levels of anxiety but there was no criterion regarding the time window between screening and participation in the laboratory, resulting in 54.4% of participants who did not meet the initial inclusion criterion anymore when they came to the laboratory. Finally, MindLight is an applied game in which the mental health aim is integrated in the story and cut scenes of the game. Thus, for participants receiving the ENT trailer it became clear while playing the game that it was aimed at reducing arousal and anxiety through relaxation, undermining their expectations that the game was a pure entertainment game.
Although MindLight has been developed for and tested for efficacy in a younger age group, we expected that the 1st year psychology students in the current study would still enjoy playing the game based on our previous experiences with an older age group. 31 In addition, recent reviews have shown that biofeedback interventions work for youth and young adults,58,59 but may be more effective for young people when the feedback is integrated in an applied game, increasing their motivation and engagement. 60 Because 1st year students often experience elevated levels of anxiety,61–63 we considered MindLight an appropriate and relevant applied game for this specific age group and to test our research questions.
Notwithstanding the aforementioned limitations and remaining questions for future research, the current study integrated research on applied games with research on nonspecific factors and suggests that promoting an applied game as a mental health or entertainment game does not influence participants' experiences and engagement with the game, regardless of participants' motivation to change and symptom severity.
Footnotes
Acknowledgments
The authors would like to thank Tiffany Tsui, Kalee De France, Owen Hicks, and Ashley Patterson for their help with the data collection, data cleaning, and preparations for data analyses. They would also like to thank Nastasia Griffioen for her feedback on earlier drafts of the article. Finally, they would like to thank all the students who participated in the study.
Author Disclosure Statement
MindLight was produced by the PlayNice Institute. I.G. is cofounder of this institute. A.W., T.H., and A.L-A. declare that they have no competing (financial) interests.
Funding Information
The authors acknowledge the financial support from the Ontario Mental Health Foundation for purchasing the equipment used in the current study. The current research was further supported by the Netherlands Organization for Scientific Research (NWO; grant number 406-16-524 for A. Wols). The funding sources had no role in the design of the study, data collection, analysis, interpretation of data, writing the article, and in the decision to submit the article for publication.
