A gamified app for supporting undergraduate students’ mental health: A feasibility and usability study

Abstract

Resilience, a person's mental ability to deal with challenging situations adaptively, is an important life skill. Supporting students in building psychological resilience and coping during crises (with the COVID-19 pandemic being a prime example) is crucial. Very few mobile applications (apps) for mental health explicitly report behavioral change techniques. Moreover, only a handful of the apps that support resilience are gamified, or use smartphone sensors readily available in modern smartphones for health self-management, or were designed for use by a nonclinical population. This study describes the design of a prototype for a gamified, theory-based mobile app that utilizes the Internet of Things to provide personalized data and enhance undergraduate students’ resilience. A total of 74 participants evaluated the prototype and completed an online questionnaire during the COVID-19 lockdowns. The questionnaire included questions examining the design's feasibility for supporting resilience and questions on the System Usability Scale evaluating its usability. Regarding the evaluation of the prototype on improving psychological resilience, positive responses (M = 3.76 out of 5, SD = 0.82) were received for all functions (goal setting for studying, socializing and physical exercise, progress monitoring using sensors or self-reporting, reflection, motivational badges). The System Usability Scale returned an evaluation score of 72.9, indicating a satisfactory degree of usability. The resilience app is a promising proof of concept. Combining Internet of Things capabilities with active user interaction while incorporating behavior change techniques in a gamified environment was well accepted by students. Implications for the design of gamified environments for well-being are drawn. Future research will empirically validate its design using quasi-experimental methods.

Keywords

Mobile apps mental health resilience building higher education COVID-19 behavioral change techniques internet of things smartphone sensors well-being prevention intervention

Introduction

During the COVID-19 pandemic, educational institutions were shut down worldwide, which impacted over 60% of students and caused a massive disruption of the education system.¹ Studies conducted since the outbreak of the pandemic uncovered a significant elevation of COVID-19-related anxiety among students in European countries.^2–6 The COVID19 pandemic is anticipated to result in long-term adverse mental health effects on the population⁷; therefore, there is an urgent need to support students’ psychological resilience. Psychological resilience refers to adapting successfully in the face of stress or adversity,⁸ overcoming daily challenges, coping with stress, and achieving goals despite obstacles.⁹ Psychological resilience is an important life skill that remains highly relevant in the post-pandemic era.

Pusey et al.¹⁰ recently reported that interactive technologies could deliver effective resilience interventions in an accessible, cost-effective, and flexible manner. Their scoping review included several types of interactive technologies used in resilience interventions, such as serious video games, virtual reality simulations, social robots, and commercial off-the-shelf video games. Their scoping review did not include mobile applications. However, mobile devices and associated software applications are fast becoming an integral part of health care, if for no other reasons than advantages of access, convenience, and cost.^11,12

Mobile applications and medical smartphone apps are being extensively used in the health domain.^13,14 As their potential usage scenarios in healthcare continue to expand, there is a concomitant need for open, accessible, and scalable digital tools, as are smartphone mental health apps.¹⁵ According to a recent systematic review of digital mental health interventions¹⁴ smartphone-based interventions were the second most common technology studied in articles included in the review, reported in 27.4% (57/208) of the articles. The focus of mobile health applications seems to be on health behavior change in physical activity,¹⁶ diet, and drug and alcohol use rather than mental health.¹⁷ According to a review conducted by Milne-Ives et al.,¹⁸ mental health seems to be the topic receiving the least attention. It is also important to note that most studies on operational mobile mental health apps address major mental health issues while prevention and well-being areas are underdeveloped.¹⁹

A literature review revealed several attempts to develop and validate resilience apps within the last five years; most efforts target adults belonging to specific professional groups.^7,20–23 For example, some resilience apps targeted mental health care providers for reducing burnout²¹ and, more recently, to support their mental health in the COVID-19 era.⁷ Other apps targeted users with significant depression symptoms²³ or targeted military personnel, veterans, and civilian first responders.²² Only two resilience apps were identified in the literature that targeted youth. The first one focused on youth who experienced sexual violence and had adverse childhood experiences^8,24 and the second one focused on student veterans with posttraumatic stress disorder (PTSD).^25,26 The effectiveness of these resilience apps was promising through small-scale studies^21,24,25 and randomized controlled trials.^20,23

Previous research suggests that gamification could increase engagement with mobile prevention interventions.²⁰ Gamification may have a positive motivational effect compared with other self-guided interventions.²⁷ Gamification was used extensively for mental health,^27–29 for example, in stress management applications.^17,30–33 However, only a few studies were found that used gamification to improve resilience in a mobile mental health intervention.^20,23,34 These studies had positive findings demonstrating effectiveness but did not address a general population.

Fleming et al.²⁹ showed that it is feasible to translate traditional theory-based and evidence-based interventions into computer gaming formats. However, most mobile health apps do not have strong theoretical foundations. Only a slim minority of developed apps are theory-based. For example, out of 50 unique apps for health behavioral change tested in the Milne-Ives et al.¹⁸ review, only a few explicitly reported the behavioral change techniques (BCTs) incorporated into the app. None of the resilience apps identified in the literature refers to behavioral change techniques, even though some refer to psychoeducational components^20,34 and emotion regulation.⁸

In recent years, research interest focused on tools for objectively measuring activity-related parameters and promoting health-related outcomes.³⁵ The role of the Internet of Things (IoT) is becoming increasingly important in an individual's daily activities. Users have engaged actively with IoT technology through various end-user devices. Smartphones can be used in an IoT setup along with different application categories, one of which is “Personal IoT” where we find an increasing number of applications targeting health and fitness, and helping to solve everyday problems for users.³⁶ The information gathered from the sensors is then used to draw useful conclusions on the user's activities and/or surroundings; such conclusions may lead to providing recommendations to a person (e.g. a slight change to lifestyle, engagement in physical exercise, etc.). Data that can be passively collected from sensors embedded in modern smartphones, (such as native GPS, and accelerometer), allow for new ways to measure psychological health. For example, data on changes in daily physical activity collected with smartphone GPS and accelerometer technology predict mood states before phone users themselves report mood changes.^11,37 Adopting IoT technologies for mental health has been slow³⁸; concerning the use of IoT technology to provide objectively measured personalized health data to users, none of the gamified apps on resilience made use of IoT in general or enabled smartphone sensors in particular.

Our study aimed to address the current and anticipated psychological impact of the COVID-19 pandemic on undergraduate students by designing and developing a mental health mobile app. A few resilience-building apps target specific professional groups’ needs.^7,20–23 The resilience app described in this study stands out for several reasons. Prevention interventions have received less attention, representing an underdeveloped area in mobile mental health apps.¹⁹ Very few mobile applications (apps) for mental health explicitly report behavioral change techniques. Very few of the apps that support resilience are gamified, use smartphone sensors (IoT), and are designed for use by a nonclinical population. The proposed prevention intervention app for resilience, named Student Stress Resilience, addresses these gaps. The study has a dual aim: (a) to describe the design of the proposed prevention intervention resilience app and (b) to evaluate its feasibility, acceptance, and usability based on its targeted future users.

Methodology

Design of the Student Stress Resilience prototype

The Student Stress Resilience app (SSResilience in short) provides specific and accessible recommendations to targeted users of a nonclinical population, based on IoT technology, as a prevention intervention. The target users are undergraduate students of a general population. Their input was sought in the design phase of the study, which implemented a participatory design approach³ following Aryana et al.'s¹⁹ recommendations for moving toward sociotechnical and open design strategies for designing mobile mental health solutions.

The core co-design team initially consisted of three members: two undergraduate students and the first author. The core co-design team identified a selection of relevant gamification elements, based on a literature review focusing specifically on the development of smartphone apps for mental health³⁹ and on interactive technologies to support emotional resilience.¹⁰ These gamification elements included: system recommendations, the use of smartphone sensors, the allocation of points, competition and collaboration among users, reflection, and goal setting.

As a second step, the gamification elements to be embedded in a gamified app for resilience were presented to 18 students (extended co-design team) who took a course on digital games in the fall semester of 2020. These students participated in a two-hour online, recorded participatory design session which took place during one of the course's regular meetings and was coordinated by the first author of the study. These 18 students evaluated the feasibility of the idea of a gamified app for resilience by providing their input individually to avoid the possibility of influencing each other's perspectives. They were invited to provide their suggestions about additional gamification elements and voice their concerns about the design of the proposed app by answering two open-ended questions.³

Students’ input was used by the core co-design team for designing the prototype of the app in AdobeXD. The following student-provided suggestions were implemented in the prototype of the app: (a) the inclusion of badges, (b) an anonymous leader board to increase competition among users, (c) the allocation of bonus points for the completion or partial completion of goals, (d) including ways to verify goal completion (through using phone sensors), and (e) the inclusion of a daily challenge (in the form of one daily goal) that can be shared. This user-centered design approach helped to better respond to undergraduates’ self-identified needs at a time of repeated lockdowns due to the pandemic. The details of this participatory design study were recently published.³

Mobile health apps often lack dynamic features that adapt based on users’ responses.⁴⁰ The Student Stress Resilience app will provide personalized recommendations based on user input, goal setting, and smartphone sensor data (IoT) to allow real-time engagement. Three goals were deemed the most relevant during pandemic lockdowns, as per WHO guidelines for mental health⁴¹: focusing on studying to maintain a healthy routine, engaging in physical exercise, and socializing (Figure 1(a)). Goal setting and progress monitoring are connected to using both self-reported data and objectively measured data through smartphone sensors (accelerometer and sound sensors). Concerning self-reported data, users can take a short, validated mental health survey that automatically calculates and interprets their level of anxiety (Figure 1(b)) to provide personalized recommendations to set and monitor goals related to studying. The validated mental health survey that was implemented in the app prototype is the Generalized Anxiety Disorder screener for the general population (GAD-7).^42,43 For objectively measured data using phone-based sensors⁴⁴ Student Stress Resilience uses the accelerometer for the user to set and monitor goals related to physical exercise and a sound sensor for the user to set and monitor goals related to socialization.

Figure 1.

Screenshot of the Student Stress Resilience prototype showing (a) goal setting, (b) self-reported anxiety symptoms for progress monitoring, (c) leaderboard, points, and ability to share progress on social networks, and (d) progress feedback and badges.

The app leverages gamification and intrinsic motivation to engage users. Users are expected to set one goal to focus on daily; the goal may refer to either studying, engaging in physical exercise, or socializing. As Van Roy and Zaman⁴⁵ suggested, to support the user's autonomy, a moderate amount of meaningful options should be provided. These researchers suggested providing users with at least one option that is meaningful and complies with their values while avoiding placing them in a dilemma by offering too many options.⁴⁵ Following this gamification heuristic, which is based on self-determination theory the design choice of offering one out of three possible goals to users was made. To support the user's feelings of competence, goals are manageable and perceived as feasible to fulfill.⁴⁵ Examples of manageable goals refer to at least 30 min of focused work on a specific assignment (for the goal of studying) or at least 30 min of walking (for the goal of physical exercise) or interacting with three people outside of the household in one day (for the goal of socializing).

Users can monitor their progress in reaching each goal by employing sensors for objective measurements and self-reflecting on their performance at the end of the day. They receive points and badges for completing challenges. To support users’ relatedness, social interaction is facilitated⁴⁵ through the users' ability to share their progress with other users. As a first stage what will be shared is a brief text outlining the goal that was achieved, for example, “I’ve completed my physical exercise goal in the SSResilience app.” Sharing results on social media for joint experiences and allowing for social pressure is expected to motivate users.⁴⁶

As shown in Figure 1, in the Student Stress Resilience app, five out of eight gamification elements, as defined by Tondello et al.,⁴⁷ were implemented to increase engagement and acceptability: (1) socialization, including social comparison, competition, leaderboards, and a connection with social networks (Figure 1(c)), (2) risk/reward, through the use of challenges, (3) customization, through the use of points (Figure 1(c)), (4) progression, through progress feedback (Figure 1(d)) on each one of three goals that can be set daily (studying, physical exercise, or socialization), and (5) incentives, through the use of badges (Figure 1(d)).

The design of the SSResilience app is theory-based, as nine behavioral change techniques, as defined by Milne-Ives et al.,¹⁸ were used: (a) action planning, (b) feedback on outcomes of behavior, (c) self-monitoring of behavior through reflection, (d) social support, (e) social comparison, (f) prompts/cues, (g) achievements, and (h) incentives.

Mobile health apps often feature dense sections of psychoeducational text that are perceived as not useful.⁴⁰ As Connolly et al.⁴⁰ caution: “Mental health researchers are attempting to squeeze complex behavioral therapies into patients’ pockets without adapting them to an app context, and it is often not working.” To overcome this limitation, the app provides specific, brief, and accessible recommendations and resources, including psychological health information from the World Health Organization, to its users using minimal text. An example of psychological health tips is the following:

The World Health Organization recommends:

1. Know that it is okay to feel how you are feeling, it is normal to experience feelings of sadness, frustration, stress.

2. Maintain a routine: start your day at about the same time each day.

3. Practice good sleep hygiene of 7–9 hours per night.

4. Connect with others by engaging in regular calls with friends and family.

Student Stress Resilience also offers a validated mental health survey, as suggested by Golden et al.⁷ The GAD-7 survey only includes seven statements (Figure 1(b)). Statements are easy to complete and can be scored automatically. Users can automatically be classified into four categories based on their score: normal anxiety (score 0–4), mild anxiety (score 5–9), moderate anxiety (score 10–14), and severe anxiety (score 15–21).⁴² Users do not however receive this classification and their calculated score is not visible to them. The calculated score is only used so that the app will automatically provide its interpretation to the user. Users only receive an interpretation of their score (e.g. “You may be experiencing some anxiety at the moment. Try a breathing exercise to relax before you study” in case their score indicates “moderate” or “severe” anxiety OR “You have a low level of anxiety which wouldn't interfere with your studying. Keep it up!” in case their scores indicate “normal” or “mild” anxiety).

The app also offers other more generic features as it generates reminders at a time chosen by the user to engage, and logs past app use. Overall, it provides a simple and intuitive interface that facilitates interactions, as suggested by Bakker et al.³⁹

Research questions

The study examined the Student Stress Resilience prototype app's feasibility, acceptance, and usability. The first two criteria for psychological health app quality suggested by Bush et al.¹¹ were used to evaluate the prototype. Specifically, (a) the user experience has been sought and tested during the design and development phase of the app with participants representative of the target audience³ and (b) the app has been pilot tested in the target environment with the target audience to examine feasibility and acceptance.

The two research questions of the study were:

- RQ1: How do students evaluate the feasibility and acceptance of the Student Stress Resilience prototype?

- RQ2: How do students evaluate the usability of the Student Stress Resilience prototype?

Participants, data collection, and ethical issues

Participants were recruited from the authors’ social and professional networks, and they were asked to voluntarily participate in the study if they satisfied two criteria: (a) they were adults (older than 18 years old) and (b) they had an undergraduate student status. Twenty undergraduate students were invited to participate as part of a class taught by the first author (convenience sampling). The response rate was 100% in the sense that all students who attended the class participated in the study. A total of 60 participants were individually invited by the second author (convenience sampling) who had a student status at the time and contacted his peers who were attending either his university or one of four different universities at the time the research study took place. University affiliation was established because the second author knew the participating students personally. Out of these 60 students, 54 responded positively. Therefore, the response rate for the second author was 90%.

In the first stage of the study, students were informed in writing about the study's objective. The study was conducted according to the World Medical Association Declaration of Helsinki. The study followed American Psychological Association (APA) ethical standards and General Data Protection Regulation (EU) 2016/679 (GDPR) guidelines, and its protocol is in accordance with the University Ethics Committee and with the country's National Bioethics Committee (Approval number: EEBK EΠ 2021.01.218). It meets ethical guidelines, including adherence to the legal requirements of the country where the study was conducted. Participants provided their consent online by selecting boxes indicating that they are adults (older than 18 years old), understand the study's objective, and voluntarily provide anonymous data. At the second stage of the study, participants were asked to follow a link that led them to a working prototype of Student Stress Resilience demonstrating its basic functionality. Participants could interact with it for as long as they wanted. At the third stage of the study, participants were asked to complete an evaluation questionnaire. The data collection period was between 31 March and 20 April 2021. It was conducted online as social distance measures were imposed on the country due to the COVID-19 pandemic.

Data sources

The questionnaire consisted of a total of 29 questions. Ten of these questions were related to the feasibility and acceptance of the application's design (RQ1), and another 10 were related to its usability evaluation (RQ2). For evaluating feasibility and acceptance an instrument was developed specifically for the needs of this study and it was not validated. For evaluating usability, a highly robust instrument was used (System Usability Scale).⁴⁸

The remaining questions were basic demographic questions examining age, gender, previous use of health apps, and frequency of use of health apps. Lastly, a question was included that indicated participants’ interest in trying the application when it would be fully developed.

Feasibility and acceptance were evaluated by asking participants to indicate how useful each specific function of the app would be for supporting resilience. The following question was used: “To what extent do you think the following design functions of the app are useful for supporting resilience during the COVID19 pandemic? Resilience is our ability to manage challenges (e.g., social distancing measures/lockdowns) adaptively.” Participants were asked to evaluate the usefulness of 10 app functions that are shown in Table 1 (in the second column “App functionality”), using a 5-point Likert scale ranging (1 = not at all useful, 2 = slightly useful, 3 = somewhat useful, 4 = very useful, 5 = extremely useful) after testing the app to familiarize themselves with each demonstrated functionality.

Table 1.

Feasibility and acceptance of 10 characteristics of app functionality for supporting resilience.

Theory-based, gamified app components	App functionality	M	SD	NPA (%, n)*
Goal setting	Studying	3.88	0.98	66.2% (49/74)
	Exercising	3.92	0.90	71.6% (53/74)
	Socializing	3.72	1.07	62.2% (46/74)
Progress monitoring (IoT)	Accelerometer	3.70	1.04	61.1% (46/74)
Progress monitoring (IoT)	Sound sensor	3.34	1.11	41.9% (31/74)
Progress monitoring (self-reported)	Anxiety test	3.89	1.02	68.9% (51/74)
Progress monitoring (self-reported)	Reflection	3.99	1.07	70.2% (52/74)
Achievements/incentives	Badges	3.81	1.11	63.5% (47/74)
Social support	Share progress	3.54	1.18	51.3% (38/74)
Social support	Leaderboard	3.47	1.09	48.7% (36/74)

*NPA: Net Percent Agreement (composite score of 4 = “very useful” and 5 = “extremely useful”).

The usability evaluation was conducted using the System Usability Scale (SUS),⁴⁸ a methodology frequently used to evaluate medical apps.⁴⁹ SUS was chosen because (a) it is a highly robust and versatile tool for usability professionals,⁵⁰ and (b) it allows the comparison of similar systems. The scale included the following 10 items, with responses graded by a 5-point Likert scale ranging from completely disagree to completely agree (Figure 2).

Figure 2.

System usability scale (SUS) for the student stress resilience prototype.

Data analysis

To analyze the first research question (RQ1), which focused on feasibility and acceptance, descriptive statistics (frequencies, percentages, Mean, SD) were run for all 10 app functions (see Table 1, column “App functions”) separately and using a composite score. A Net Percent Agreement (NPA) was calculated, which consisted of the composite score of participants’ answers of 4 = very useful and 5 = extremely useful.

For the analysis of the second research question (RQ2), which focused on usability, the procedure for calculating usability evaluation scores proposed by the author of the instrument SUS was followed.⁴⁸ Specifically, for odd-numbered items, one was subtracted from the user response. For even-numbered items, the user responses were subtracted from 5. This scaled all values from 0 to 4 (with four being the most positive response). The converted responses for each user were added up, and the total was multiplied by 2.5. This converted the range of possible values from 0 to 100 instead of 0 to 40. An average SUS score was calculated for all participants. The SUS score was then converted into a percentile rank and a letter grade from A to F, according to the norms proposed by Sauro.⁵¹

Results

Demographic data

A total of 74 adult participants (44 male and 30 female) with an average age of 22 years old (M = 21.86, SD = 1.78, min = 18, max = 24) tested and then evaluated Student Stress Resilience online due to social distance measures and university closures imposed due to the COVID-19 pandemic. Participants came from five different universities with the majority of them coming from the university of the authors. Most of the participants (58.1%, 43/74) were users of similar applications for health in the past, while less than half of the participants (45.9%, 34/74) were current users of a health app. The majority of current users of such applications (56.4%, 22/34) did not use them often. Approximately one in four participants (28.2%, 11/39) used a health app once or twice per month, and the same percentage of participants (28.2%, 11/39) reported that they used their health app rarely. Only 36% of participants (35.9%, 14/39) used their health app daily or almost daily.

RQ1 feasibility and acceptance of the Student Stress Resilience prototype

Regarding the evaluation of Student Stress Resilience in terms of its feasibility for psychological resilience, positive responses (approximating four out of five, on average) were received for the perceived usefulness of all functions (Table 1), while the composite evaluation score of all functionality examined was M = 3.76 (SD = 0.82). The highest score was reported for progress monitoring based on self-reported measures, specifically the functionality that refers to self-reflecting at the end of the day (M = 3.99, SD = 1.07) to accumulate points based on desired behavior and actions. The self-administered anxiety test was also evaluated with a high score (M = 3.89, SD = 1.02). This can also be observed using the NPA score, where we see that approximately 70% of participants found self-reported progress monitoring as very useful or extremely useful.

The lowest score was reported for the use of IoT as an objective way to measure and monitor progress, specifically the functionality that refers to the use of a sound sensor (M = 3.34, SD = 1.11) as a proposed way to measure surrounding sound and subsequently receive points concerning achieving the goal of socializing with other people. This finding may indicate participants’ potential lack of familiarity with or lack of interest in using sensor-based data for health purposes, even though the use of an accelerometer was seen more positively (M = 3.70, SD = 1.04) compared to the use of a sound-sensor. This may be attributed to the value that students place on the goal of increasing their physical exercise, which was evaluated with the second-highest score in reference to descriptive statistics (M = 3.92, SD = 0.90) and the highest score (71.6%, 53/74) in reference to NPA.

The vast majority of participants (93.2%, 69/74) stated that they would be interested in trying the application when it is completed, a finding that shows high acceptability. A statistically significant, moderate positive correlation (Pearson's r = 0.434, p = 0.00) was found between the overall evaluation of the application's functions and participants’ interest to try out the app once it is fully developed. This indicates that the higher the students’ feasibility evaluation of the SSResilience prototype, the higher their interest in trying out the fully functional app.

A statistically significant positive correlation was also found between the overall feasibility evaluation of Student Stress Resilience and participants’ age. A statistically significant, positive, moderate relationship (Pearson's r = 0.28, p = 0.016) was found between participants’ age (M = 21.86, SD = 1.78) and their evaluation of the application's functions (M = 3.76, SD = 0.82). This indicates that older students who were potentially more mature and more experienced in overcoming university life challenges were more inclined to appreciate the potential value of a gamified application for resilience.

Additional analyses were run to examine trends with respect to potential in-group comparisons concerning gender and whether participants were current users of health apps. Concerning gender comparisons, there were no significant differences with respect to the feasibility evaluation of Student Stress Resilience (t₇₂ = −0.75, p > 0.05) when an independent samples t-test was run to compare the mean of male students (M = 3.70, SD = 0.87) to the mean of female students (M = 3.84, SD = 0.76). Using the same analysis, no significant differences (t₇₂ = 0.88, p = 0.38) were identified between the feasibility evaluation means of current users of health apps (M = 3.86, SD = 0.79) and non-users of health apps (M = 3.69, SD = 0.85).

RQ2 usability evaluation of the Student Stress Resilience prototype

The SUS was utilized to evaluate the application's usability. To provide an answer to RQ2, the average scores of the SUS scale provided by 74 students were calculated. The app received a satisfactory usability score. The raw average SUS score was 72.9, which is in the 70% percentile rank, corresponding to the category “B” of the scale, a result that shows a satisfactory degree of usability.

Figure 3 shows how the percentile ranks (y-axis) are associated with SUS scores (x-axis) and letter grades.⁵⁰ This process is similar to “grading on a curve” based on the distribution of all scores; for example, a raw SUS score of 74 converts to a percentile rank of 70%. A SUS score of 74 has higher perceived usability than 70% of all products tested.⁵⁰

Figure 3.

The raw average system usability scale (SUS) score of the Student Stress Resilience prototype, based on a sample of 74 participants.

Females evaluated the usability of the app (M = 77.58, SD = 12.50) significantly higher (t₇₂ = −2.19, p = 0.032) compared to males (M = 69.66, SD = 16.91). No significant differences were found (t₇₂ = 1.73, p = 0.088) between the way usability was evaluated by current users of health apps (M = 76.25, SD = 15.72) and non-users of health apps (M = 70.00, SD = 15.23) even though current users evaluated the app higher compared to non-users.

Discussion

This study described the design of Student Stress Resilience, the prototype for a gamified, theory-based, mental health mobile application to support undergraduate students’ resilience in a post-pandemic future. With undergraduates feeling the pandemic stress,^2,3,6 it makes sense to offer them resources close to home and relevant to their lives and studies. Previous mental health app research highlighted the need for app personalization and customization.⁶¹ Common barriers that affect user engagement with regard to a user's uptake and sustained interactions with digital mental health interventions include a lack of personalization.¹⁴ Kenny et al.¹² also agreed with the need for telemental health interventions to be tailored and personalized to target specific types of users. The Student Stress Resilience app attempted to overcome these barriers¹⁴ and address this need.¹² It stands out in that it provides specific, accessible, and personalized recommendations to undergraduate students by integrating data collected by smartphone sensors (accelerometer and sound sensor) and user-provided data that comes from a validated mental health survey^7,42 and is interpreted for them in simple terms.

Previous research suggests that gamification, turning intervention content into a game format, is effective in increasing user intention to use mHealth apps,⁵² especially in the healthy and younger groups.⁵³ Mobile apps using gamification can be a valuable and effective platform for well-being and mental health interventions and may enhance motivation and reduce attrition.²⁰ Framed as games, apps become potentially powerful tools to promote well-being.²³ Gamified smartphone-based self-help programs and prevention and early-stage mobile interventions for supporting resilience have shown to be effective and engaging for mental health at least for the resilience of significantly depressed individuals²³ of men³⁴ and of young professionals.²⁰ The SSResilience app stands out due to its targeted nature, in that it is a gamified mobile prevention intervention addressed to and designed for use by a nonclinical undergraduate student population.

Nowadays, different sensors and IoT devices can readily be used to passively collect various health data—including physical activity, social interaction, sleep, and mobility patterns—and make inferences about mental and physical health.^18,54 The potential of using IoT in the context of mental health applications has been hardly harnessed.³⁸ The IoT promises many benefits to streamlining and enhancing health care delivery to proactively predict health issues.⁵⁵ As Kelly et al.⁵⁵ noted, IoT can improve the accessibility of preventative public health services but this is still in theory. The smartphone is considered to be at the heart of a growing universe of connected devices and sensors.³⁶ In this study, the focus was on smartphones which have impressive sensing and processing capabilities that make them an ideal tool for collecting data. This study attempted to take advantage of IoT capabilities that are easily accessible by utilizing smartphone-based sensors in a prevention intervention for a mental health issue.

Mobile applications using smartphone sensors to provide personalized data have been connected with behavioral change, but there is insufficient empirical evidence to verify this connection. The Milne-Ives et al.¹⁸ review found little evidence of changed behavior or health outcomes due to the use of mobile health apps. As Abedtash and Holden³⁵ put it, it seems that “devices alone or with minimal behavioral change support are insufficient to change health-related outcomes.” This further reinforces the importance of designing evidence-based apps with strong theoretical foundations to examine their impact on adopting and maintaining healthy behaviors. As Hoffmann et al.³⁰ noted, app designers do not exploit the potential of combining gamification techniques with behavior change theory. The proposed resilience app combines gamification techniques with behavior change theory and IoT functionality and addresses gaps identified in the literature.

The study attempted to examine the feasibility and acceptability of the Student Stress Resilience prototype. The composite feasibility evaluation score of all functionality examined (M = 3.76, SD = 0.81) is relatively positive considering the fact that health app use was not very popular among the sample of undergraduate students, as less than half of them were current users of health apps, and of those, the majority did not use them daily. Specific functionality allowing progress monitoring using self-reported measures (such as reflection at the end of the day and self-reported anxiety) was seen more positively than the objective measurement of user activity using IoT sensors (sound sensor and accelerometer). As only quantitative data were collected in this study there are no data on the participants’ motivation behind these ratings. Some hypotheses can be made based on the literature. A lower preference for objective measurements made possible through IoT could perhaps be attributed to users’ perceptions of viewing health apps’ influence as paternalistic,⁴⁶ which is a hypothesis expressed in the literature but not yet confirmed by empirical studies. It may also relate to users’ social resistance to IoT technologies, which are not yet mainstream,⁵⁶ or to users’ hesitancy to provide access to their personal data.³ Although the sound sensor has been less accepted than other features, it will be maintained in the next version of Student Stress Resilience to assess better whether it can positively impact usage, as its ability to measure ambient noise represents a unique feature not encountered in resilience apps.

Some studies on mental health apps indicate that smartphone devices are a promising self-management tool for mental health, specifically for reducing symptoms of depression^57,58 and anxiety.⁵⁹ Other studies argue that there remains debate around whether these apps have demonstrated high efficacy⁶⁰ and that their effectiveness and potential for sustained use remain uncertain.⁴⁰ Developing and evaluating a functional resilience app will shed light on this debate.

Some implications for the design of gamified environments for health can be drawn from this study. The user experience has to be sought and evaluated during the design and development phase of gamified apps with participants representative of the target audience, as shown by Nicolaidou et al.³ Prototypes of gamified apps that may not be fully functional have to be pilot tested in their targeted environment with their targeted audience to examine feasibility and acceptance of the idea before engaging in full development. This may significantly impact financial investments and reduce the cost of developing features that the targeted audience may not need or value. Integrating IoT technology in gamified apps for health seems to be a promising approach as it may provide data that can be valuable for several stakeholders, including (a) the users themselves, as they can receive personalized recommendations based on their data and changing trends over time, (b) app developers, as they can potentially use these data for app improvement, and (c) researchers, as they can use these data to understand better which behavior change techniques may positively impact user behavior more effectively and efficiently.

Limitations

The study had several limitations. The study did not include testing with a functional app, its sample was small and it was reached using convenience sampling rather than random sampling. The instrument that was used for evaluating the app's feasibility and acceptance was developed specifically for the needs of this study and it was not validated. Participants were self-selected, and only quantitative data were collected to evaluate the prototype's feasibility, acceptance, and usability. These limitations are expected to be addressed in future work.

Future work

Given the study's small sample size with self-selected participants and no testing with a functional app, further investigations can focus on developing a fully functional app followed by a randomized controlled trial. As the evidence base is still uncertain regarding the effectiveness and usability of mobile health apps,⁴⁰ our future studies will involve users in interacting with the app to evaluate its efficacy. As the overwhelming majority of participants in this study expressed an interest in trying out a functional app, a larger group of interested participants is expected to participate in future studies. Participating students can be divided into experimental and control groups to investigate how their resilience scores change, measured with a validated resilience scale (e.g. CD-RISC)⁹ when they use the Student Stress Resilience alpha app (experimental) and no app (control) for the same amount of time (∼3 months). In this way, Student Stress Resilience is expected to address the urgent need for engaging, appealing, and effective mental health interventions which reach large numbers of people, as suggested by Fleming et al.²⁹

Findings from trials and user reviews of some commercially available programs suggest that applied gaming approaches can be engaging; however, many analyses do not report engagement or ongoing use.²⁹ Findings of Connolly's et al.⁴⁰ narrative review that aimed at understanding factors influencing sustained use of mental health apps suggest that for apps to be successful, they must be advantageous over alternative tools, relatively easy to navigate, and aligned with users’ needs, skills, and resources. The proposed Student Stress Resilience app has some unique features that make it stand out. Future research will measure its efficacy, scalability, and sustainability, or potential for sustainable use, in real-life conditions.

Conclusion

Student Stress Resilience is a promising proof of concept for those who wish to build a mobile health app to support their employees, communities, or others in managing and improving mental well-being. It is a novel tool aiming to broadly offer mental health support, which shows potential for scalability for the larger student nonclinical population. The proposed app may be used either by students struggling in a post-pandemic era or by self-motivated individuals looking to find creative ways to maintain and improve healthy balanced lifestyle habits. The latter includes a healthy balance between responding to university studying responsibilities, engaging in physical exercise, and maintaining social interactions, among other activities.

Footnotes

Acknowledgements

The authors would like to thank all undergraduate students who voluntarily participated in this study. The authors would also like to thank Christos Christodoulou for his technical support with respect to the Student Stress Resilience prototype development.

Contributorship

IN and LA researched literature, conceived the study, and gained ethical approval. LA developed the prototype of Student Stress Resilience and he was responsible for participant recruitment, data collection, and analysis. LL had a consulting role in the prototype development. IN supervised the study and wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The study was conducted according to the World Medical Association Declaration of Helsinki. The study followed APA ethical standards and GDPR (EU) 2016/679 guidelines, and its protocol is in accordance with the Cyprus University of Technology Ethics Committee and with the Cyprus National Bioethics Committee (Approval number: EEBK EΠ 2021.01.218). It meets ethical guidelines, including adherence to the legal requirements of the country where the study was conducted.

Funding

The authors received no financial support for the research, and/or authorship of this article. The publication fee is funded by the Cyprus University of Technology Open Access Author Fund. The future work described in this article will be funded by a start-up grant awarded to the first author by the Cyprus University of Technology (2022–2024).

Guarantor

IN.

ORCID iD

Iolie Nicolaidou

References

Alqahtani

Rajkhan

. E-learning critical success factors during the COVID-19 pandemic: a comprehensive analysis of e-learning managerial perspectives. Educ Sci 2020; 10: 216.

Kaparounaki

Patsali

Mousa

DPV

, et al. University students’ mental health amidst the COVID-19 quarantine in Greece. Psychiatry Res 2020; 290: 113111.

Nicolaidou

Aristeidis

Christodoulou

, et al. Co-creating a gamified app for enhancing students’ emotional resilience in times of crisis (COVID-19). INTED2021 Proceedings 2021: 4169–4175. https://doi.org/10.21125/inted.2021.0850

Nicolaidou

Stavrou

Leonidou

. Building primary-school children’s resilience through a web-based interactive learning environment: quasi-experimental pre-post study. JMIR Pediatrics and Parenting 2021; 4: e27958.

Patsali

Mousa

Papadopoulou

. University students’ changes in mental health status and determinants of behavior during the COVID-19 lockdown in Greece. Psychiatry Res 2020; 292: 113298.

Solomou

Constantinidou

. Prevalence and predictors of anxiety and depression symptoms during the COVID-19 pandemic and compliance with precautionary measures: age and sex matter. Int J Environ Res Public Health 2020; 17: 4924.

Golden

Zweig

Danieletto

, et al. A resilience-building app to support the mental health of health care workers in the COVID-19 era: design process, distribution, and evaluation. JMIR Formative Research 2021; 5: e26590.

Mushquash

Pearson

Waddington

, et al. User perspectives on a resilience-building app (JoyPop): qualitative study. JMIR Mhealth Uhealth 2021; 9: e28677.

Campbell-Sills

Stein

. Psychometric analysis and refinement of the Connor–Davidson resilience scale (CD-RISC): validation of a 10-item measure of resilience. J Trauma Stress 2007; 20: 1019–1028.

10.

Pusey

Wong

Rappa

. Resilience interventions using interactive technology: a scoping review. Inter Learn Environ 2020: 1–16. https://doi.org/10.1080/10494820.2020.1772837

11.

Bush

Armstrong

Hoyt

. Smartphone apps for psychological health: a brief state of the science review. Psychol Serv 2019; 16: 188–195.

12.

Kenny

Dooley

Fitzgerald

. Feasibility of “CopeSmart”: a telemental health app for adolescents. JMIR Ment Health 2015; 2: e4370.

13.

Buechi

Faes

Bachmann

, et al. Evidence assessing the diagnostic performance of medical smartphone apps: a systematic review and exploratory meta-analysis. BMJ Open 2017; 7: e018280.

14.

Borghouts

Eikey

Mark

, et al. Barriers to and facilitators of user engagement with digital mental health interventions: systematic review. J Med Internet Res 2021; 23: e24387.

15.

Torous

Wisniewski

Bird

, et al. Creating a digital health smartphone app and digital phenotyping platform for mental health and diverse healthcare needs: an interdisciplinary and collaborative approach. J Technol Behav sci 2019; 4: 73–85.

16.

Lee

Frey

Min

, et al. Usability inquiry of a gamified behavior change app for increasing physical activity and reducing sedentary behavior in adults with and without autism spectrum disorder. Health Informatics J 2020; 26: 2992–3008.

17.

Tozzi

Nicolaidou

Galani

, et al. Ehealth interventions for anxiety management targeting young children and adolescents: exploratory review. JMIR Pediatrics and Parenting 2018; 1: e7248.

18.

Milne-Ives

Lam

De Cock

, et al. Mobile apps for health behavior change in physical activity, diet, drug and alcohol use, and mental health: systematic review. JMIR Mhealth Uhealth 2020; 8: e17046.

19.

Aryana

Brewster

Nocera

. Design for mobile mental health: an exploratory review. Health Technol 2019; 9: 401–424.

20.

Litvin

Saunders

Maier

, et al. Gamification as an approach to improve resilience and reduce attrition in mobile mental health interventions: a randomized controlled trial. PLOS ONE 2020; 15: e0237220.

21.

Wood

Prins

Bush

, et al. Reduction of burnout in mental health care providers using the provider resilience mobile application. Community Ment Health J 2017; 53: 452–459.

22.

Kizakevich

Eckhoff

Lewis

, et al. Biofeedback-assisted resilience training for traumatic and operational stress: preliminary analysis of a self-delivered digital health methodology. JMIR Mhealth Uhealth 2019; 7: e12590.

23.

Roepke

Jaffee

Riffle

, et al. Randomized controlled trial of SuperBetter, a smartphone-based/internet-based self-help tool to reduce depressive symptoms. Games Health J 2015; 4: 235–246.

24.

MacIsaac

Mushquash

Mohammed

, et al. Adverse childhood experiences and building resilience with the JoyPop app: evaluation study. JMIR Mhealth Uhealth 2021; 9: e25087.

25.

Reyes

Muthukumar

Bhatta

, et al. Promoting resilience among college student veterans through an acceptance-and-commitment-therapy app: an intervention refinement study. Community Ment Health J 2020; 56: 1206–1214.

26.

Reyes

Bhatta

Muthukumar

, et al. Testing the acceptability and initial efficacy of a smartphone-app mindfulness intervention for college student veterans with PTSD. Arch Psychiatr Nurs 2020; 34: 58–66.

27.

George

Jameson

Michael

, et al. Assessing the efficacy of a self-administered treatment for social anxiety in the form of a gamified mobile application: a pilot study. J Technol Behav Sci 2021; 6: 124–136.

28.

Bordini

Münscher

Baumgartner

, et al. Strangers in a strange land: designing a mobile application to combat loneliness and isolation among foreign university students. J Technol Behav sci 2021; 6: 81–87.

29.

Fleming

Bavin

Stasiak

, et al. Serious games and gamification for mental health: current status and promising directions. Front Psychiatry 2017; 7: 215.

30.

Hoffmann

Christmann

Bleser

. Gamification in stress management apps: a critical app review. JMIR Serious Games 2017; 5: e7216.

31.

Nicolaidou

Tozzi

Kindynis

, et al. Development and usability of a gamified app to help children manage stress: an evaluation study. Italian Journal of Educational Technology 2019; 27: 105–120.

32.

Nicolaidou

Tozzi

Antoniades

. A gamified app on emotion recognition and anger management for pre-school children. Int J Child Comput Interact 2022; 31: 100449.

33.

Nicolaidou

Tozzi

Kindynis

, et al. An Interactive Storytelling Game for Mobile Devices for Children’s Stress Management. In: European Conference on Games Based Learning [Internet]. Reading, United Kingdom: Academic Conferences International Limited; 2017 [cited 2021 Feb 16]. p. 967–971. Available from: https://www.proquest.com/docview/1967764022?pq-origsite=gscholar&fromopenview=true

34.

Cheng

VWS

Davenport

Johnson

, et al. Naturalistic evaluation of a sport-themed mental health and wellbeing app aimed at men (MindMax), that incorporates applied video games and gamification. Internet Interv 2020; 20: 100306.

35.

Abedtash

Holden

. Systematic review of the effectiveness of health-related behavioral interventions using portable activity sensing devices (PASDs). J Am Med Inform Assoc 2017; 24: 1002–1013.

36.

Khaddar

MAE

Boulmalf

. Smartphone: The ultimate IoT and IoE Device [internet]. In: Nawaz Mohamudally (Ed) Smartphones from an Applied Research Perspective. IntechOpen, 2017 [cited 2022 Mar 30]. Available from: https://www.intechopen.com/chapters/56113

37.

Saeb

Zhang

Karr

, et al. Mobile phone sensor correlates of depressive symptom severity in daily-life behavior: an exploratory study. J Med Internet Res 2015; 17: e4273.

38.

Gutierrez

Rabbani

Ajayi

, et al. Internet of things for mental health: open issues in data acquisition, self-organization, service level agreement, and identity management. Int J Environ Res Public Health 2021; 18: 1327.

39.

Bakker

Kazantzis

Rickwood

, et al. Mental health smartphone apps: review and evidence-based recommendations for future developments. JMIR Ment Health 2016; 3: e4984.

40.

Connolly

Hogan

Shimada

, et al. Leveraging implementation science to understand factors influencing sustained use of mental health apps: a narrative review. J Technol Behav sci 2021; 6: 184–196.

41.

Mental health and psychosocial considerations during the COVID-19 outbreak [Internet]. [cited 2022 Mar 30]. Available from: https://www.who.int/publications-detail-redirect/WHO-2019-nCoV-MentalHealth-2020.1

42.

Löwe

Decker

Müller

, et al. Validation and standardization of the generalized anxiety disorder screener (GAD-7) in the general population. Med Care 2008; 46: 266–274.

43.

Spitzer

Kroenke

Williams

JBW

, et al. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006; 166: 1092–1097.

44.

Ogbuabor

. Human Activity Recognition for Healthcare using Smartphones. In: Proceedings of the 2018 10th International Conference on Machine Learning and Computing [Internet]. New York, NY, USA: Association for Computing Machinery; 2018 [cited 2022 May 31]. pp. 41–46. (ICMLC 2018). Available from: https://doi.org/10.1145/3195106.3195157

45.

van Roy

Zaman

. Why Gamification Fails in Education and How to Make It Successful: Introducing Nine Gamification Heuristics Based on Self-Determination Theory. In: Ma

Oikonomou

(eds) Serious Games and Edutainment Applications: Volume II [Internet]. Cham: Springer International Publishing, 2017 [cited 2021 Nov 4], pp. 485–509. Available from: https://doi.org/10.1007/978-3-319-51645-5_22

46.

Kühler

. Exploring the phenomenon and ethical issues of AI paternalism in health apps. Bioethics 2022; 36: 194–200.

47.

Tondello

Mora

Nacke

. Elements of Gameful Design Emerging from User Preferences. In Association for Computing Machinery; 2017 [cited 2021 Sep 29]. Available from: https://uwspace.uwaterloo.ca/handle/10012/13714.

48.

Brooke

. SUS: A “Quick and Dirty” Usability Scale. In Usability Evaluation in Industry (pp. 2–8). CRC Press, 1996.

49.

Alamoodi

Garfan

Zaidan

, et al. A systematic review into the assessment of medical apps: motivations, challenges, recommendations and methodological aspect. Health Technol 2020; 10: 1045–1061.

50.

Bangor

Kortum

Miller

. An empirical evaluation of the system usability scale. International Journal of Human–Computer Interaction 2008; 24: 574–594.

51.

Sauro

. SUStisfied? Little-Known System Usability Scale Facts User Experience Magazine [Internet]. 2011 [cited 2022 Jan 18]. Available from: https://uxpamagazine.org/sustified/.

52.

Wang

Fan

Zheng

, et al. The impact of gamification-induced users’ feelings on the continued use of mHealth apps: a structural equation model with the self-determination theory approach. J Med Internet Res 2021; 23: e24546.

53.

Lee

. Mobile healthcare applications and gamification for sustained health maintenance. Sustainability 2017; 9: 772.

54.

Ghosh

Dey

. “Sensing the Mind”: An Exploratory Study About Sensors Used in E-health and M-health Applications for Diagnosis of Mental Health Condition. In: Chakraborty

Ghosh

Ravi

Shelke

(eds) Efficient Data Handling for Massive Internet of Medical Things: Healthcare Data Analytics [Internet]. Cham: Springer International Publishing, 2021 [cited 2022 Apr 8], pp. 269–292. Available from: https://doi.org/10.1007/978-3-030-66633-0_12

55.

Kelly

Campbell

Gong

, et al. The internet of things: impact and implications for health care delivery. J Med Internet Res 2020; 22: e20135.

56.

Kim

Park

. Understanding social resistance to determine the future of internet of things (IoT) services. Behav Inf Technol 2022; 41: 547–557.

57.

Firth

Torous

Nicholas

, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017; 16: 287–298.

58.

Six

Byrne

Tibbett

, et al. Examining the effectiveness of gamification in mental health apps for depression: systematic review and meta-analysis. JMIR Ment Health 2021; 8: e32199.

59.

Firth

Torous

Nicholas

, et al. Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. J Affect Disord 2017; 218: 15–22.

60.

Chandrashekar

. Do mental health mobile apps work: evidence and recommendations for designing high-efficacy mental health mobile apps. Mhealth 2018; 4: 6.

61.

Strecher

McClure

Alexander

, et al. The role of engagement in a tailored web-based smoking cessation program: randomized controlled trial. J Med Internet Res 2008; 10: e1002.