Towards a self-applied,mobile-based geolocated exposure therapy software for anxiety disorders: SyMptOMS-ET app

Abstract

Objective

While exposure therapy (ET) has the potential to help people tolerate intense situation-specific emotions and change avoidance behaviours, no smartphone solution exists to guide the process of in-vivo ET. A geolocation-based smartphone software component was designed and developed to instrumentalize patient guidance in in-vivo ET and its psychological validity was assessed by a group of independent psychology experts.

Methods

A team of computer scientists and psychologists developed the ET Component for in-vivo ET using geolocation-based technology, following the process-centred design methodology. The ET Component was integrated into the SyMptOMS-ET Android application, which was developed following the co-design methodology. Next, nine independent psychology experts tested and evaluated the ET Component and the SyMptOMS-ET app in the field, following the think-aloud methodology. Participants also completed the Mobile Application Rating Scale (MARS) instrument to quantitatively evaluate the solutions.

Results

We present the SyMptOMS-ET app’s main features and the ET Component exposure workflow. Next, we discuss the feedback obtained and the results of the MARS instrument. Participants who tested the app were satisfied with the ET Component during exposure scenarios (score of $μ$ 4.32 out of 5 [ $σ$ 0.28] on MARS quality aspects), agreed on the soundness of the theoretical foundations of the solutions developed (score of $μ$ 4.57 [ $σ$ 0.48] on MARS treatment support aspects), and provided minor think-a-loud comments to improve them.

Conclusions

The results of the expert evaluation demonstrate the psychological validity of the ET Component and the SyMptOMS-ET app. However, further studies are needed to discern the acceptability and efficacy of the mHealth tool in the target population.

Keywords

Anxiety in-vivo exposure therapy smartphone mHealth ecological momentary assessment ecological momentary intervention just-in-timeintervention

Introduction

From 1990 to 2019 the disability-adjusted life years associated with anxiety disorders in 204 countries were 28.68 million (95% uncertainty interval: 19.86, 39.32).¹ Following the global outbreak of COVID-19, the number of cases of anxiety disorders has dramatically increased by more than 25% worldwide, totalling more than 76 million additional cases worldwide.² Unfortunately, despite the high prevalence and costs of anxiety disorders and even though efficient screening tools and evidence-based psychological treatment exist, there exists a treatment gap: an important percentage of individuals are underdiagnosed and undertreated. For example, Ford et al.³ estimated a treatment gap of 57.5% for generalized anxiety disorder. Even though the pandemic did not trigger this situation – it accelerated it⁴ – it inevitably led to greater demand for therapeutic assistance in already globally saturated and under-resourced health systems. For example, the WHO reports 13 therapists for every 100,000 inhabitants,⁵ with countries barely dedicating a 2% of their budgets to these treatments.²

To help alleviate this treatment gap, digital tools have been proposed (e.g. Agyapong et al.⁶), showing promising initial results in helping trained and untrained people access mental healthcare resources.^7,8 Some of the advantages of these tools include low costs, widespread availability and low stigma.⁹ From the patient’s perspective, the smartphone is the preferred digital medium¹⁰ to provide mental health treatment. In this sense, mHealth tools have not only been seen as a mechanism to support therapy but also as a way to make it more accessible and effective,¹¹ with potential cost reductions¹² and faster access to relevant information.¹³

Anxiety disorders in adults include specific phobias (e.g. claustrophobia), panic, agoraphobia, social anxiety, and generalized anxiety disorder.¹⁴ Common in these disorders are the negative reactions to the emotional experience (i.e. anxiety) and the intention to avoid any stimulus that triggers anxiety symptoms, which tend to be provoked by specific places (situational avoidance) where an intense emotional reaction occurred or where affected people think it is more likely to happen.¹⁵ For such disorders, exposure therapy (ET) is an effective tool for increasing tolerance, reducing anxiety levels, and changing maladaptive beliefs.^16–19 The cognitive and behavioural mechanisms of change in ET explain the efficacy of this component, as irrational beliefs and avoidance or safety behaviours that may be maintaining the anxiety disorder are modified.²⁰ Additionally, there are neurobiological mechanisms that support ET, such as a reduction in the activity of brain areas related to emotional fear responses (e.g. insula, amygdala, and anterior cingulate cortex) and an increase in the prefrontal cortex, which regulates emotional responses.²⁰

The American Psychology Association defines ET as ‘a psychological treatment that was developed to help people confront their fears’.²¹ Fear is a natural emotion that prepares our body for something we perceive as harmful. Although it is generally regarded as negative, fear serves a useful purpose and helps us to survive. Nevertheless, the verb ‘perceive’ is key here: anxiety disorders originate from the perception that something is harmful, but in reality, it is not. This distortion could have been generated by a past negative or traumatic experience that should not be extrapolated to similar situations.

In these cases, ET is a well-proven means for psychologists to help their patients tolerate intense emotions such as anxiety and reduce avoidance and discomfort towards stimuli that trigger an emotional response. Together with the patient, the psychologist builds an exposure hierarchy, which contains a list of stimuli that the patient avoids. They can be places, situations, objects, people, animals, or anything else that makes patients feel uncomfortable and triggers an intense anxiety response. Depending on the patient’s assessed condition and the perceived usefulness, convenience and feasibility of the treatment, a therapist establishes an exposure plan, determining how the patient progresses from one situation to another and how the exposure is conducted (means of exposure or delivery strategies). During treatment, patients must be repeatedly exposed to the agreed situations, hereby progressively becoming accustomed to them and, therefore, advancing in the exposure hierarchy.²²

There are several means to carry out an exposure: in-vivo, imaginal, based on virtual reality (VR) or interoceptive.²³ This work focuses on the first, in-vivo exposure, in which patients are exposed to situations feared in real life. For example, going to a high bridge (fear of heights), attending a party (social anxiety), open spaces (agoraphobia), being far away from a hospital or home (panic disorder), or with spiders in captivity (phobia). During in-vivo exposure, patients can benefit from the presence of their psychologist, to guide them and provide appropriate instructions and feedback based on patients’ perceived feelings and reactions. Although it is highly effective, considering that exposure sessions last on average between 60 and 90 min, these consume a lot of time and resources of psychologists,²⁴ who are already over-demanded and understaffed.² Therefore, the premise of our research is that, by harnessing the full potential of smartphones, it should be possible to guide ET practice remotely and in a fully instrumented manner, relegating the presence of therapists to only the most challenging and complex situations.

In the literature, there are examples of mHealth applications containing ET components. However, they generally act as a diary rather than as a guide during exposure. For example, in cognitive-behavioural therapy (CBT) Assistant²⁵ patients create their exposure hierarchies and fill in a diary through the app to report what happened after exposure to any given situation. Zemedy²⁶ offers ET as a self-applied therapy. While the app has more advanced features, such as a chatbot and multimedia content, the behaviour of the ET component is comparable to that of CBT Assistant (i.e. an exposure diary). Other examples of self-applied therapy apps add reminders for patients to begin their exposures (e.g. SmartCAT²⁷ and MindClimb²⁸), similar to how PE Coach²⁹ does for in-person sessions. Some applications allow ET to be performed using smartphone-based augmented reality (AR).^30,31 Yet, the smartphone is merely a technological aid for the AR and the therapist must still physically accompany and guide the patient through the exposure. These apps offer some peripheral assistance for (in-vivo) ET, but no guidance or assistance to the patient during exposure. Therefore, to the best of our knowledge, there is no mHealth application that offers patient guidance during in-vivo exposure, as confirmed by Denecke et al.³² in a 2022 literature review.

In this article, we present a fully instrumented geolocation-based ET software component (ET Component) that systematically monitors patients (ecological momentary assessment) and guides them through the entire exposure process (ecological momentary intervention) during the psychological treatment for anxiety disorders. Disorders for which the SyMptOMS-ET application may be particularly suitable are those in which the location of the places recommended for exposure sessions plays a relevant role. Within the range of anxiety disorders, specific phobias, agoraphobia or social phobia hereby stand out as especially relevant to be considered for the app. The ET Component has been co-designed and developed by a multidisciplinary team of computer scientists and psychologists (the authors). We here marry the technical and computational skills necessary for reliable application implementation with psychological expertise for modelling the complicated workflow and psychological guidance of ET. The ET Component was integrated into the SyMptOMS-ET app, which was conceived and designed as a complement to traditional therapy sessions. The SyMptOMS-ET app and its ET Component are presented in ‘Results’. Both tools were validated through simulations to ensure technical correctness and robustness, followed by field testing with a committee of independently recruited psychology experts to determine compliance with the theoretical principles of ET and to evaluate our digital adaptation of ET. The results of the Mobile App Rating Scale (MARS) instrument³³ are also presented in ‘Results’, followed by a discussion based on the MARS results and expert feedback in ‘Discussion’.

Methods

This study falls into the category of healthcare system improvement studies.³⁴ The theory behind the improvement (co-design) along with its technical materialization (development) are first introduced in the ‘Design and development’ section. Both were done in iterations. Next, in the ‘Evaluation process’ section, a full evaluation is pursued, including software engineering methods to verify the technical correctness of the SyMptOMS-ET app combined with an expert-based mixed-methods pre-clinical evaluation methodology, following recommended practices.³⁵ Finally, supporting tools for statistical analysis are indicated in the ‘Statistical analysis’ section.

Design and development

The design and development process was iteratively carried out in four phases. The first two focused on the design of the ET Component, including the adaptation of the ET process (‘Phase 1 – ET Component design specification’) and content (‘Phase 2 – Psychoeducational and intervention content generation’) for delivery via smartphones, while the other two covered the SyMptOMS-ET app design, integrating and complementing the ET Component with a visual and interactive design (‘Phase 3 – SyMptOMS-ET app design’), and providing an implementation (‘Phase 4 – SyMptOMS-ET app development’).

Phase 1 – ET component design specification

Our multidisciplinary team of three psychologists and four computer scientists brainstormed to mutually understand each other’s field and co-design a fully instrumentalized mobile ET solution. The design consisted of two key components: (a) formalizing and adapting the classical ET process to be suitable for automated, mobile delivery and (b) identifying technical (smartphone) features to support the adapted ET process. Whereas the general framework for exposure is well defined in the literature, in practice, therapists use many visual and implicit cues from the patient (e.g. facial expressions and hesitation) to guide an in-vivo exposure. Capturing this implicit knowledge and devising a digital equivalent, that is, determining possible execution paths throughout an exposure, was critical to success. Considering the exposure as a closed process, which the patient must carry out but resists doing so, we used the process-centred design methodology³⁶ as a guiding framework. Originating from the field of business applications, process-centred design involves information technologists and business process experts who digitize business processes and hereby prioritize company goals over employee comfort to achieve greater productivity or performance. Applied to our healthcare scenario, we used it to adapt the ET process to a software format, whereby correct execution of the ET process determines the success of the treatment,³⁷ even though patients instinctively prefer to avoid exposure to feared situations. Using online whiteboards and collaborative writing tools and through multiple iterations, we thus formalized the ET process, basing ourselves on the existing literature,^22,24,38,39 and hereby focusing on the overall ET workflow and the codification of informal cues, as follows.

The ET process was first limited to a timeframe (60 to 90 min) based on the existing literature recommendations²⁴ and divided into three phases: pre-, during- and post-exposure. For each phase, entry, internal decision and exit points were identified. For example, for pre-exposure, a patient entering the vicinity of the exposure area is an entry point, whereas a patient hesitating and leaving the vicinity of the area (abandonment) or entering the area and starting the exposure (initialize exposure) are two exit points. Examples of internal decision points are sending a motivational message or psychoeducational content, allowing the patient a brief escape (i.e. exiting and re-entering the exposure area shortly after), extending the duration of exposure, etc. Finally, in close relation to the internal decision points (and, consequently, the possible execution paths), for each phase, the optimal points of interaction with the patient were determined. For example, in pre-exposure, prompt a baseline assessment at the start of exposure or encourage starting after 10 min of inaction. The frequency of recurrent patient interactions is key: they should be fine-grained enough to allow for meaningful decision points, but not too frequent as to disturb or distract the patient from emotional responses. To determine the optimal frequency, an experiment was undertaken with healthy subjects, whereby different notification frequencies were tested, determined by the Fibonacci sequence in minutes. Recurring notifications of 1, 2 and 3 min were reported as exhausting, while every 5 and 8 min were reported as acceptable. Although a 5-min frequency provided more detailed feedback, it was observed that people were more aware of the phone, being a possible source of distraction during exposure. For this reason, the team’s psychologists determined the frequency of 8 min was optimal. A detailed description of the ET process and its digital adaptation is given in ‘Results’.

With the formalized ET process in place, the design focused on determining technological means to execute the process on a mobile device. Four dimensions were considered: presence detection w.r.t. the exposure area, subjective assessment of the patient’s state, change detection in the patient’s behaviour, and reaction to detected and reported patient’s states. For presence detection, it was determined that the phone’s location (i.e. Global Positioning System (GPS) coordinates) combined with geofencing analysis is suitable for detecting the relative position of patients to exposure areas. For subjective assessment, essential to guide exposure, active sensing was chosen to capture the levels of anxiety discomfort and tolerance. Rather than in-app assessments, whereby the patient keeps the app open and is alerted when the assessment is requested, we opted for the smartphone system notifications to deliver in-app assessments. This allows the patients to close the app (i.e. it runs as a background service) and put the phone aside between evaluations, reducing the risk of distractions resulting from phone use. For change detection, a mix of technologies was selected, using physical activity detection mechanisms available on smartphones in combination with phone location acquisition (with geofencing analysis) to detect significant movements. Finally, for reacting to the detected and reported changes in the patient’s condition, we favoured a traditional rule-based system over machine learning techniques, due to its greater explicability and predictability, and also due to the lack of data on previous exposures conducted under similar conditions.

Phase 2 – Psychoeducational and intervention content generation

Based on the ET literature,^22,38,39 the team of psychologists initially agreed upon and proposed the psychoeducational content to be included, along with its adaptation to the smartphone format. The rest of the team reviewed the content to ensure that it was sufficiently understandable for a non-expert audience. The content was organized into three groups: always-available psychoeducational content, notification content, and timely delivered intervention content (linked to the notifications). The same collaborative tools that were used for the ET Component design phase were employed, where the initial document proposal was collaboratively refined.

For always-available psychoeducational content, the existing theoretical ET background and the experience of the psychologist team were considered. It contains all relevant theoretical concepts, written to be understood by non-experts and viewed at any time by the patient. For the notification content, the challenge was to fit the title and content of the notifications to a line of text on the phone’s lock screen, making them understandable at a glance without expanding them and, at the same time, imperative and empathic to the patient’s feelings. Lastly, to specify timely delivered intervention content (linked in notifications), anticipating the possible patient context before each intervention (i.e. the patient’s state and that of the surroundings) was crucial to elaborate personalized content that helped the patient in every envisioned situation through step-by-step guidelines. Supplemental material includes a document with both original and translated versions of all the psychoeducational, notification and timely delivered intervention contents.

Phase 3 – SyMptOMS-ET app design

The next step was to tie the above pieces together, presenting them through a smartphone application interface: the SyMptOMS-ET app design. In this phase, a selection of required features was agreed on to be included in the smartphone app, wrapping the ET Component, to deliver the greatest possible value to the patient. Here, the co-design methodology⁴⁰ was key for initially conducting a brainstorming session to identify relevant app features, followed by a cycle of visual and interaction design sessions. In addition to the previous online collaborative tools, the Figma tool⁴¹ for designing high-fidelity wireframes and no-code screen flows was key to facilitating the flow of information in the co-design methodology.

Next, one of the team members, a design expert, created the first high-fidelity proposal of the app’s screens and its navigation flow using Figma. The proposal was then shared and refined in collaboration with the rest of the team members. As a result, an interactive graphic representation of the screens and navigation flow of the final mobile app was obtained. The supplemental material includes a dossier listing all application screens and features.

Phase 4 – SyMptOMS-ET app development

Taking the above design and functional specifications, the behaviour-driven development (BDD) methodology⁴² was used to develop the ET Component and the container SyMptOMS-ET app. Extracted user stories were added to a Kanban backlog for iterative development, to periodically present functionality increments to all team members and incorporate feedback as the app development progressed.⁴³ Technology-wise, the Android platform was chosen due to its support for background context sampling and arbitrary code execution, while TypeScript⁴⁴ with the Angular web framework⁴⁵ and the NativeScript⁴⁶ mobile development framework was the application technology stack of choice.

The first challenge to address was the documented issue of unreliable execution of background tasks (e.g. sensor measurements).⁴⁷ We systematically explored and tested the behaviour of various Android devices (i.e. different brands and operative system (OS) versions) and existing solutions (e.g. Android WorkManager, Firebase JobDispatcher and Evernote Android-Job), but none showed reliable behaviour over prolonged time periods. Therefore, we defined a set of guidelines for improving systematic background scheduling on Android devices and coded these guidelines into a software library called NativeScript Task Dispatcher (NTD), which harnesses the Android system’s low-level scheduling mechanisms (alarms). NTD was extensively evaluated experimentally to demonstrate the reliability and efficiency of its background task scheduling mechanism.⁴⁸ Exploiting the systematic sensor measurements and event-driven nature of the NTD, we thus composed background execution workflows such as those required by the ET Component, for example, to detect user movement and trigger actions for systematic collection of location data, which was then analyzed to calculate relative proximity to an exposure area (geofencing) and, if required, deliver a baseline (pre-exposure) assessment. Widely used tasks were generalized, implemented, packaged, and made available as modules of the context-aware AwarNS Framework,⁴⁹ to serve not only the ET Component but also other types of mobile (health) applications. The AwarNS Framework implements the sense–analyse–act paradigm on top of the NTD for context sampling and information processing and provides primitives and modules for context-aware applications. Remarkable features are support for location-based and inertial sensors (e.g. gyroscope and accelerometer) and active sensing; custom, geofencing and machine learning data analysis; and persistence and notification delivery. AwarNS also allows the implementation of custom, application-specific tasks used in the SyMptOMS-ET app, for example, custom tasks in the ET Component such as handling geofence change events and processing patient answers to assessments.

In the SyMptOMS-ET app, interoperable software layers enabled the communication between Angular and the AwarNS Framework in such a way that crucial user interface (UI) actions were propagated as AwarNS events for background processing (e.g. assessment answers and manual exposure finalization), and AwarNS events triggered UI changes (i.e. a notification tap opening a specific UI view). The SyMptOMS-ET app interacts with the SyMptOMS server to enable secure user authentication, patient profile retrieval, and data upload for remote analysis and visualization (through a web dashboard for therapists). Figure 1 schematically shows these software components and the interaction between them, which constitutes the SyMptOMS-ET app (including the ET Component) that is further described in ‘Results’ Section.

Figure 1.

SyMptOMS-ET mobile app software components interaction diagram. At the bottom, the NativeScript Framework is an abstraction layer on top of the mobile OS SDK (Android). The NTD, AwarNS and Angular (UI) are built on top of NativeScript, to access OS features. The NTD schedules background tasks. AwarNS defines concrete tasks for context sampling. The ET Component extends AwarNS with tasks specific to the exposure process. Angular uses NativeScript for native UI drawing and AwarNS to start background workflows, react to notification interactions and recover data analysis results. AwarNS and Angular interact with the SyMptOMS Server for secure user authentication, patient profile retrieval and data upload. OS: operative system; SDK: source development kit; NTD: native task scheduler; UI: user interface.

Evaluation process

The correct functioning, robustness, and technical specifications of the SyMptOMS-ET app were constantly verified during its development, using automated software testing tools according to the BDD methodology. Manual testing was done in the lab at the end of each Kanban iteration before each feature increment was shown to the rest of the team. Simulation tests were performed in which, for example, environment-related events (e.g. exposure area detection, patient physical activity changes, etc.) were replaced by UI actions that generated equivalent events (available only in the app’s development version).

Following the recommendations prior to a clinical trial,³⁵ field tests were carried out with a set of independent psychology experts to validate the process-centred design approach and evaluate the decisions made regarding the digital adaptation of the exposure technique in the ET Component. Participant recruitment was conducted by the team of psychologists according to the following eligibility criteria: people with experience in the treatment of at least one patient, fluent in Spanish (the language used in the app) and with intermediate knowledge of English (to understand the MARS instrument questions). Participants were contacted by email and were delivered a written informed consent form via Google Forms, which also served to collect sociodemographic data. All participants signed the form. The study was approved by the Ethics Committee of the Universitat Jaume I of Castellón (expedient no. CD/78/2022). As a result of the recruitment, nine independent psychology experts tested and validated the functionality of the ET Component by using the SyMptOMS-ET app. Participants belonged to three public universities and one private psychology office, all located in Spain. For demographics and other professional characteristics of the participants, see Table 1.

Table 1.

Demographic and professional characteristics of the nine participants.

Participant	Genre	Age	Educational level	Clinical expertise	Patients treated	Times applying ET	ICT knowledge [1–5]	Use of ICT in therapy
1	Male	37	PhD	6–9 years	15+	15+	5	15+
2	Female	34	PhD	6–9 years	15+	15+	3	15+
3	Female	40	PhD	10–15 years	15+	3–5	4	6–9
4	Male	28	MD	<1 year	1–2	1–2	3	–
5	Female	44	MD	15+ years	15+	15+	4	15+
6	Female	35	PhD	6–9 years	15+	10–15	4	15+
7	Female	27	PhD	<1 year	10–15	–	3	3–5
8	Male	45	MD	3–5 years	15+	6–9	3	3–5
9	Female	25	MD	<1 year	15+	1–2	2	1–2

ET: exposure therapy; ICT: information and communication technology.

Exposure places were agreed upon between the team of psychologists and the participants and added to the SyMptOMS-ET app by the authors. A Xiaomi Poco X3 Pro (M2102J20SG) on Android 11 was used for testing, with the SyMptOMS-ET app preinstalled and configured. The smartphone’s built-in screen recording tool was configured to use the phone’s microphone to capture users’ speech. All the data captured by the SyMptOMS-ET app was uploaded to a remote server for further analysis and post-processing. Before starting, participants were asked to follow the think-aloud method,⁵⁰ to make their thoughts explicit while using the app and record their reflections. During field testing, participants assumed the role of patients and, along with a set of action instructions (focusing on the what rather than how), followed three typical exposure scenarios at designated places nearby their workplaces:

The patient successfully completes an exposure. During exposure, the patient reports a peak anxiety level, but can significantly reduce it, that is, a drop of 3 units on a 0 to 10 Subjective Units of Distress Scale (SUDS),⁵¹ before reaching 60 min of exposure. Therefore, the exposure finishes without further extension.

The patient already feels comfortable with the dreaded situation. The patient shows signs of low discomfort during the pre-exposure evaluation and also during the first couple of assessment reports. Therefore, the exposure finishes early, and it is considered successful.

The patient’s discomfort is greater than indicated when building the hierarchy. The patient initially suffers from severe anxiety and prematurely leaves the area, therefore abandoning the exposure.

Upon completion, participants were given a graphic dossier detailing all the app’s features to explore (available as supplemental material) and a paper copy of an adapted version, tailored to the needs of the study following the instructions present in the template of the evaluation instrument, of the MARS³³ (validated by Terhorst et al.⁵²) to be filled in in order to evaluate the SyMptOMS-ET app and the ET Component. The field tests yielded the following results: a dataset containing all the quantitative and qualitative data collected by the app, screen interaction and audio recordings, notes on think-a-loud thoughts, and the MARS instrument responses. After the tests, relevant highlights were extracted from the screen and audio recordings, and the results of the MARS forms were statistically analysed as described next. The exposure instructions, the graphical dossier and the adapted MARS instrument are all available as supplemental material.

Statistical analysis

The quantitative scores obtained through the MARS evaluation instrument were anonymized and statistically analysed using R 4.3,⁵³ Quarto 1.4⁵⁴ and R Studio 2024.04.1.⁵⁵ Following the instructions present in the MARS instrument, mean ( $μ$ ) and standard deviation ( $σ$ ) for each dimension evaluated were calculated. From the aggregated measures of each dimension, $μ$ and $σ$ were again computed on each MARS section (A–F). Finally, overall $μ$ and $σ$ were obtained using the aggregate values from sections. Complementary boxplots were also generated (see ‘After exposure: Expert ratings’).

Results

Before exposure: SyMptOMS-ET app set up

The resulting SyMptOMS-ET app is a wrapper of the fully instrumented geolocation-based ET Component, designed as a complementary instrument to face-to-face therapy, guiding patients through in-vivo exposures. As such, the app is installed and configured during an in-person therapy session, after introducing the ET treatment and its benefits to the patient. At this point, the therapist and the patient agree on an exposure plan, that is, the exposure hierarchy and its situations. Items in the exposure hierarchy are georeferenced situations, that is, in this case, specific places, areas or zones represented by their geographic perimeters (geographic polylines).

The app connects to a web dashboard, where therapists can create patient profiles (app accounts). Within a profile, therapists can set up exposure hierarchies whose items (places such as parks, squares, shopping malls, bridges, public transport stations, etc.) directly cause an intense anxiety response in the patient. Therapists can also generate one-time passwords, which are provided to patients for use in the mobile app, thus linking a patient profile to a particular app installation. For privacy reasons, patient profiles only contain their medical file identifier and details of their exposure hierarchies, excluding any personal information.

Once the app is configured, the ET Component works silently in the background according to the workflow shown in Figure 2. It periodically captures the patient’s location. When near an exposure area (100 m or less to the edge of the area), patients are asked to confirm their intention to begin exposure. After confirmation, they are prompted with a baseline assessment (i.e. pre-exposure discomfort [0 (none) to 10 (maximum)], tolerance [0 (none) to 10 (a lot)] and beliefs of what will happen during the exposure [free text input]), along with additional instructions for starting the exposure. Once the app detects the patient entering the exposure area, it delivers a text guide so the patient knows what to expect during the exposure. All prompts and instructions provided by the app before, during and after the exposure occur in the form of notifications that link to static content or interactive views of the app.

Figure 2.

Pre-exposure flowchart.

During exposure: SyMptOMS-ET app

During exposure, patients can visit the places of their exposure hierarchy for time-limited in-vivo exposures, whereby the SyMptOMS-ET app guides them through periodic assessments via notification-based (mini-)questionnaires and timely, location-based interventions such as action guidelines and psychological content. The overall exposure workflow is shown in Figure 3. Internally, the app ‘monitors’ the patient’s location through background GPS tracking, and stores their responses to the recurring assessments to adapt the app’s behaviour accordingly. For example, patients receive regular notifications (every 8 min) to evaluate current anxiety discomfort [0–10] and tolerance [0–10] levels. These collected values are combined with pre-exposure values to detect ongoing trends, which are used by the ET Component to shorten the duration of the exposure or deliver specific coping instructions, affirmatively supportive (reward) or motivational (reinforcement) messages as follows (see Figure 4):

If the patient’s discomfort level reported at the baseline assessment is 3 or less and that trend prevails during the next two assessments during exposure, the situation is considered overcome and the patient is rewarded and told they can leave the exposure area. The exposure is considered successfully concluded.

During exposure, after a consecutive series of reports of high discomfort (8+ three times, 9+ two times or a single 10), the app considers that the patient suffers from high anxiety and delivers coping instructions. Such instructions are only delivered once during a series of high anxiety reports, to avoid overwhelming the patient.

During exposure, every two assessment reports, and whenever the last discomfort level is <8, the patient receives an affirmative support message. Messages are randomly chosen from a preestablished list to avoid repetition.

During exposure, every two assessment reports, and whenever the last discomfort level is $\geq$ 8 and as long as it is not under a consecutive series of high anxiety reports (see second bullet above), the patient receives a motivational message.

Figure 3.

General exposure flowchart. Further details of exposure flowcharts are shown in Figures 4 to 7.

Figure 4.

Evaluation of patient responses to assessments delivered during the exposure flowchart.

After one hour of exposure (Figure 5), the ET Component determines whether or not it can be concluded based on the patient’s evolution:

If the patient’s last reported discomfort level is <5, the patient is rewarded and told they can leave the exposure area. The exposure is considered successfully concluded.

If the patient’s last reported discomfort level is >5, it is compared to the highest reported value, and only if there is a reduction of 3 or more units (e.g. started with an 8, finished with a 5), the patient is given a set of guidelines to consider for the next exposure and is told they can leave. The exposure is considered successfully concluded.

Otherwise, the patient is given a set of guidelines to cope with the anxiety and told to stay 15 min more, extending the total exposure duration to 75 min.

Figure 5.

Global exposure evaluation after fixed elapsed times to limit its duration flowchart.

If the exposure time extends for 15 min, the ET Component re-evaluates the patient’s evolution. This time, if the patient’s last reported discomfort level is <8, the patient is rewarded for the effort and given a series of guidelines for the next exposure; otherwise, another 15 min extension is granted. Once the second time extension has elapsed, the patient is rewarded for staying 30 min longer in the exposure area and is advised to practice more emotional regulation skills, which are part of the psycho-education provided in the accompanying face-to-face therapy sessions, with general guidelines being included in the always-available content of the app. The exposure is considered successfully concluded.

In addition to the patient’s responses to periodical assessments, the ET Component also analyses changes in the patient’s location. Specifically, the same mechanism for detecting proximity or presence in an exposure area is used to detect if the patient leaves the area (i.e. crosses the exposure boundary) or escapes from the exposure (i.e. moves far away from the exposure boundary). When escaping (Figure 6), the app asks the patient to confirm whether the escape was intentional or not (e.g. poor GPS accuracy and some event beyond the control of the patient occurred). If the escape is confirmed, the app offers psychoeducational content on the topic of emotional avoidance to convince the patient to return to the exposure area. If despite this, the patient still abandons the exposure area, then they are asked to provide feedback on the reason and the exposure is labelled as terminated without success.

Figure 6.

Exposure abandon detection mechanism flowchart.

Note that this feedback is also requested if the patient manually ends the exposure and the last recorded discomfort level is <8. Otherwise, guidelines are delivered to prevent sensibilization and dissuade the patient from leaving (see Figure 7). The final instance for requesting this feedback is when the patient does not intend to perform an exposure while near an exposure area (see Figure 2).

Figure 7.

Behaviour of always-accessible user actions during the exposure flowchart.

Upon successful completion of an exposure, the app offers two final questions: confirmation or denial of pre-exposure beliefs and attention or distraction to what was happening during the exposure. Through the main application view (progress), the patient can consult the evolution of the anxiety level during the exposure and compare it with previous exposures (in the same place and other places). All the computations required before, during and after the exposure are performed on the mobile device, therefore exposures can be conducted offline. Figure 8 shows an illustration that summarizes the SyMptOMS-ET app features described above, and others such as the panic button and textual psychoeducational content and coping strategies, among others (see supplemental material for a detailed list of features and content). The SyMptOMS-ET app is open source and publicly accessible.⁵⁶

Figure 8.

SyMptOMS-ET smartphone application feature highlights.

After exposure: Expert ratings

Regarding the specific aspects of the ET Component, all the participants were impressed with how responsive the app was during the exposure to what they reported through the app interface (e.g. discomfort and tolerance level) and what they did (regarding changes in position relative to the exposure area). This is later confirmed by the average score obtained in section B (Functionality) of the MARS instrument. Overall, there were no negative comments regarding how the ET Component behaves and the content of the messages it delivers to the patient. Only one suggestion was given to improve the ET process: provide specific additional content in case the patient reports a high level of anxiety (8 or more) during the baseline assessment. Regarding the SyMptOMS-ET app, some suggestions to improve or add new features were:

Add an extra question after the exposure, inquiring into the use of safety behaviours (e.g. going with a relative, an amulet, etc.).

Include illustrations and animations in affirmative support and motivational messages.

Improve the presentation of the content, splitting textual content into different screens to space its delivery and using multimedia content to ease delivery.

Emphasize more that the phone must be put aside during the exposure and only be used to respond to the delivered notifications.

The evaluation conducted using the MARS instrument reported the following results. In the ‘App Classification’ section, participants mainly identified the app focuses as: ‘Anxiety/Stress’ first, ‘Behavior Change’ second and ‘Reduce negative emotions’ third. In terms of theoretical background, ‘Monitoring/Tracking’ was the first pick, followed by ‘Assessment’, ‘Feedback’, ‘Information/Education’, ‘Advice/Tips/Strategies/Skills training’ and ‘CBT – Cognitive (thought challenging)’. Moving to the ‘App Quality Ratings’, overall, the app was rated with a mean 4.32 score out of 5 (

σ = 0.28

), which is the average of the mean engagement (Section A), functionality (Section B), aesthetics (Section C) and information (Section D) scores. Details on the results obtained, jointly with the ‘App Subjective Quality’ (Section E) and ‘App-specific’ (Section F) scores, are summarized in Figure 9 and described below.

Figure 9.

Results obtained from the answers to each of the six sections of the Mobile Application Rating Scale (MARS) instrument. All items are rated using a Likert scale from 1 to 5, with two exceptions: (E2) Expected usage times in the next 12 months, with possible answers: 1 (none), 2 (1–2 times), 3 (3–10 times), 4 (10–50 times) and 5 (>50 times); and (E3) willing to pay for its use, with possible answers: 1 (no), 3 (maybe) and 5 (yes).

All the data captured by the app during the tests and the detailed anonymized MARS instrument responses are publicly available.⁵⁷

Discussion

Principal findings

The results of the test conducted by independent psychology experts showed a good reception of the SyMptOMS-ET app and the ET Component. The overall lack of negative comments indicated that the co-design and process-centred methodologies were successful in producing a result that generally matched the experts’ expectations for an app implementing an ET component. Using quantitative and qualitative methods provided a better understanding of the quantitative scores and deeper insight into possible improvement areas.⁵⁸

Firstly, the experts correctly understood and identified the focus and theoretical background of the app, as indicated by the positive evaluation of the MARS’s Information (D) and App-specific (F) sections (see Figure 9). This implies that the relevant clinical and research background was adequately incorporated, improving over other applications evaluated in the literature,⁵⁹ and that all the critical paths of the ET process were covered and that the envisaged therapeutic ET strategies were appropriately implemented and adapted to the smartphone capabilities. Involving experts from the beginning set a difference in achieving this scoring in these sections (4.53 in ‘Information’ and 4.57 in ‘App-specific’), which tend to score the lowest in the studies where they are reported.^59,60 Secondly, the overall appreciation for the app was high, with an average app quality score (4.32 out of 5.00), involving both application aspects and the ET Component, with a strong agreement among all the participants ( $σ = 0.28$ ). This suggests that aspects related to content (ET Component), presentation and usability (SyMptOMS-ET app) received high user satisfaction. In this sense, MARS app quality scores generally show high inter-rater reliability and internal consistency levels.⁵⁸ Nevertheless, specific presentation and usability issues were still subject to improvement, particularly some suggestions for adding multimedia motivational messages and content to the SyMptOMS-ET, which corresponds with one of the design features to improve user engagement identified through a systematic review and thematic synthesis by Wei et al.⁶¹ In what follows, we discuss each section of the MARS instrument.

For application Engagement (Section A, Figure 9), the quantitative results are consistent with the qualitative feedback obtained from the think-aloud method, which indicated a general appreciation for the good functioning of the location-based ET Component, and the capacity of the app to mimic therapist-guided exposures, with detailed support for 15 decision points and 136 paths through the clinical exposure.⁵⁸ This has been before a point of struggle with existing apps targetting anxiety disorders.⁵⁹ Only the ‘customization’ aspect scored relatively lower (mean of 3.4 out of 5). Even though patients generally value customization options or they even may be essential,⁶² this lower score corresponds with the general tendency as found by a recent review of mental health apps, where ‘customization’, ‘interactivity’ (Section A), ‘credibility’ and ‘goals’ (Section D) were generally found to be the lowest scoring quality items (Sections A to D).⁶³ Even though the current implementation of the ET process allows customization of the exposure hierarchy and its places (coordinates, detection, and vicinity radius) for each patient, we acknowledge that the defined ET process in terms of exposure decision points and paths is currently closed for further adaptation/customization. This is in line with recent meta-reviews of mHealth solutions in which app content is personalized more frequently than app functionalities.⁶⁴

For Functionality (Section B, Figure 9), based on experts’ qualitative feedback and comments, we conclude that the main reason behind the high score in ease of use, navigation and gestural design was the notification-based interaction design, which, unlike traditional apps, attracted attention and directly prompted the patient to perform certain actions, which is indeed suggested by a recent research community call to develop better mHealth solutions that include personalisation techniques and tools to improve feedback to users as well as engagement.⁶⁴ Experts did not make any further (negative) comments about the lower performance rating during the think-aloud method, but from observation during the test, noticeable pauses were discerned between a notification tap and the app screen displaying the corresponding content, which might be a possible explanation. Code inspection revealed that this delay was caused by an external library used to send notifications, which was subsequently replaced. Other studies were rated as having poor performance due to serious technical issues such as app freezing and long app loading,^65,66 which is not the case for the SyMptOMS-ET app. Indeed, its functionality mean (4.42) is consistent with other studies in which users reported no technical problems when using the app.⁶⁷

For the Aesthetics scores (Section C, Figure 9), the ratings on layout, graphics, and visual appeal reflected that the collaborative design methodology produced generally understandable and visually pleasing UI compositions, in line with the app’s purpose. Nevertheless, the think-aloud method yielded some suggestions for improvement regarding the user interface, such as including graphics/multimedia in content (instead of only text) and emphasizing certain instructions, which align with the advice of Knitza et al.⁶⁸ in the development of mHealth apps for other health conditions. Here, the mixed-methods approach provided further insight into an otherwise high average score (4.15), by knowing exactly which UI interface aspects could be improved (e.g. to include illustrations, animations, split content, complement with videos, etc.).⁵⁸

Concerning the Information scores (Section D, Figure 9), we consider the obtained results a logical consequence of our iterative co-design, process-centred methodology, where we worked together with domain experts to gradually come to a novel, state-of-the-art delivery of well-proven ET therapeutical content. This is a significant improvement to other previously evaluated applications aimed at anxiety disorders, which scored lowest in the Information and Engagement sections.⁵⁹ However, there is room for improvement in visualizing information (e.g. videos and animations), as suggested by the experts’ comments.

Moving on to the Subjective quality evaluation (Section E, Figure 9), results are generally favourable, even though therapists think there is still room for improvement, as previously discussed. This is also confirmed by the feedback provided during the think-aloud method, where participants shared possible improvement points.⁵⁸ Note that participants rated the app and the ET Component together, as there was no possibility to separate the ET process from the app.

Overall, the score obtained in app-specific aspects (Section F, Figure 9) showed that external experts considered the SyMptOMS-ET app and the ET Component promising for patients with place-specific situational anxiety. Here, involving psychology experts from the beginning was key for making a difference when comparing the results obtained by the SyMptOMS-ET app with those obtained by other digital health applications, commercially developed or developed without this close technician–psychologist relationship, and evaluated using the MARS instrument as well.^59,60,69

Limitations

Despite the good results achieved by the SyMptOMS-ET app and the ET Component during the evaluation, this study is subject to some limitations. Technically speaking, the ET Component is not fully independent and can currently not be directly included in any custom app. It builds upon the NTD and AwarNS framework, and we recommend researchers who wish to use it to use the open-source SyMptOMS-ET app⁵⁶ as a template to build their own applications. Hereby, any contextual-aware feature from the underlying AwarNS Framework⁴⁹ can be reused. Next, from a psychological point of view, the ET Component is capable of handling the most common exposure scenarios; however, the therapist’s assistance is still needed for exceptional cases such as extreme anxiety or comorbid conditions. For those cases, the SyMptOMS-ET app offers the panic button feature to communicate with the therapist.

Regarding the evaluation procedure, only experts residing in Spain participated in the study. Therefore, their opinions could be influenced by cultural biases, having only dealt with patients from this country. Their opinions could also have been influenced by the specific characteristics of the Spanish National Health System, which is currently saturated and needs these types of tools to alleviate the burden on its professionals.

Finally, it needs to be mentioned that this study, according to recommended practice,⁷⁰ employed independent psychologists to evaluate the app, both from the viewpoint of the therapist and enacting the patient’s role. Patients are to be involved in the next step, conducting clinical trials to further assess the acceptability, usability and effectiveness of the app.

Conclusions

In this article, we presented the SyMptOMS-ET app along with its built-in ET Component, which fully instrumentalizes ET based on passive (i.e. geolocation through GPS) and active sensing (i.e. use prompts for anxiety level discomfort and tolerance). Its functionality and content are supported by well-established ET theory and literature, as recognized by an external committee of psychology experts who tested the app in the field and validated the adaptation of the therapy to a smartphone-based digital medium.

The SyMptOMS-ET app conveniently wrapped the ET Component to enable self-guided in-vivo exposures. With it, therapists configure the SyMptOMS-ET app with place hierarchies for their patients before exposure. During exposure, patients can spatially explore their treatment exposure places through the app interface, which mimics the traditional exposure hierarchy. When visiting a place, the app guides patients through the in-vivo exposure, periodically prompting and evaluating their emotional state and intervening when necessary. Exposures are limited to 60 min, which can be extended up to 90 min depending on how the patient’s condition evolves during the exposure. After exposure, patients can check their progress by inspecting graphs of their previous exposures. The app also offers other treatment-relevant functionalities, such as psychoeducational content and a panic button that can optionally be activated. As Denecke et al.³² recently stated, the future of cognitive behavioural therapy in mobile mental health apps is the integration and testing of psychological theories to improve the impact of mobile health apps and adherence to mobile mental health interventions. In line with this vision, the SyMptOMS-ET app systematically monitors a patient (momentary ecological assessment) and guides them through the entire exposure process (momentary ecological intervention), fully respecting the theoretical foundations of ET and adapting its principles to the needs of the smartphone-based monitoring and delivery tools.

As future work, the next logical step is to carry out user acceptance tests with patients to detect areas of improvement from the patient’s perspective and compare them with those identified by experts, as preparatory steps before evaluating efficacy through clinical trials.

Finally, beyond the design and development of the SyMptOMS-ET app (and other mHealth apps), we can summarize our experiences into the following two recommendations for mobile mental health developers: (a) participation of mental health experts in the design process is essential to successfully translate traditional mHealth processes to technology-based processes, to understand and codify the intrinsic implicit cues used by therapists during interventions, and to adapt psychological content to the target device, (b) careful consideration and alignment of the smartphone’s advanced features and computational possibilities with the key characteristics of the intervention has the potential to significantly enhance the treatment delivery.

Supplemental Material

sj-zip-1-dhj-10.1177_20552076241283942 - Supplemental material for Towards a self-applied, mobile-based geolocated exposure therapy software for anxiety disorders: SyMptOMS-ET app

Supplemental material, sj-zip-1-dhj-10.1177_20552076241283942 for Towards a self-applied, mobile-based geolocated exposure therapy software for anxiety disorders: SyMptOMS-ET app by Alberto González-Pérez, Laura Diaz-Sanahuja, Miguel Matey-Sanz, Jorge Osma, Carlos Granell, Juana Bretón-López and Sven Casteleyn in DIGITAL HEALTH

Footnotes

Acknowledgments

We thank all the clinical psychologists who evaluated the SyMptOMS-ET app for making this study possible.

Code availability

The resulting code of the SyMptOMS-ET app and the ET Component is available with a permissive license at Zenodo.⁵⁶

Contributorship

All authors researched literature and conceived the study. AGP, LDS, JO, JBL and SC were involved in protocol development. AGP, CG and SC gained ethical approval. JO and JBL were in charge of participant recruitment. AGP, MMS and CG performed the data analysis. AGP wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Data availability

The anonymized responses to the MARS instrument form and the data collected and uploaded by the app during the experiment are openly available at Zenodo.⁵⁷

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The Ethics Committee of the Universitat Jaume I of Castellón approved this study (expedient no. CD/78/2022).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: AGP and MMS were funded by the Spanish Ministry of Universities (grants FPU17/03832 and FPU19/05352). This work was supported by grant PID2020-120250RB-I00 (SyMptOMS-ET) funded by MICIU/AEI/10.13039/501100011033, and by grants PID2022-1404475OB-C21 (SyMptOMS-JIT) and PID2022-1404475OB-C22 (SyMptOMS-JIT) funded by MICIU/AEI/10.13039/501100011033 and by ERDF/EU.

Guarantor

SC.

ORCID iD

Alberto González-Pérez

Supplemental material

Supplemental material for this article is available online.

References

Yang

Fang

Chen

, et al. Global, regional and national burden of anxiety disorders from 1990 to 2019: results from the global burden of disease study 2019. Epidemiol Psychiatr Sci 2021; 30: e36.

World Health Organization. World mental health report: Transforming mental health for all. Geneva: World Health Organization, 2022.

Ford

Campion

Chamles

, et al. “You don’t immediately stick a label on them”: a qualitative study of influences on general practitioners’ recording of anxiety disorders. BMJ Open 2016; 6: e010746.

Santomauro

Herrera

AMM

Shadid

, et al. Global prevalence and burden of depressive and anxiety disorders in 204 countries and territories in 2020 due to the COVID-19 pandemic. Lancet 2021; 398: 1700–1712.

WHO. Mental health atlas 2020. Geneva: World Health Organization, 2021.

Agyapong

VIO

Hrabok

Vuong

, et al. Closing the psychological treatment gap during the COVID-19 pandemic with a supportive text messaging program: protocol for implementation and evaluation. JMIR Res Protoc 2020; 9: e19292.

Feijt

De Kort

Bongers

, et al. Mental health care goes online: practitioners’ experiences of providing mental health care during the COVID-19 pandemic. Cyberpsychol Behav Social Netw 2020; 23: 860–864.

Naslund

Shidhaye

Patel

. Digital technology for building capacity of non-specialist health workers for task-sharing and scaling up mental health care globally. Harv Rev Psychiatry 2019; 27: 181.

Firth

Torous

Nicholas

, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017; 16: 287–298.

10.

Granger

Vandelanotte

Duncan

, et al. Is preference for mHealth intervention delivery platform associated with delivery platform familiarity? BMC Public Health 2016; 16: 1–7.

11.

Price

Yuen

Goetter

, et al. mHealth: a mechanism to deliver more accessible, more effective mental health care. Clin Psychol Psychother 2014; 21: 427–436.

12.

Iribarren

Cato

Falzon

, et al. What is the economic evidence for mHealth? A systematic review of economic evaluations of mHealth solutions. PLoS ONE 2017; 12: e0170581.

13.

Kim

Marcusson-Clavertz

Yoshiuchi

, et al. Potential benefits of integrating ecological momentary assessment data into mHealth care systems. Biopsychosoc Med 2019; 13: 1–6.

14.

American Psychiatric Association. Diagnostic and statistical manual of mental disorders: DSM-5. Washington, DC: American Psychiatric Association, 2013.

15.

Bullis

Boettcher

Sauer-Zavala

, et al. What is an emotional disorder? A transdiagnostic mechanistic definition with implications for assessment, treatment, and prevention. Clin Psychol Sci Pract 2019; 26: 20.

16.

Carpenter

Andrews

Witcraft

, et al. Cognitive behavioral therapy for anxiety and related disorders: a meta-analysis of randomized placebo-controlled trials. Depress Anxiety 2018; 35: 502–514.

17.

Kaczkurkin

Foa

. Cognitive-behavioral therapy for anxiety disorders: an update on the empirical evidence. Dialog Clin Neurosci 2015; 17: 337–346.

18.

Schröder

Wrona

Müller

, et al. Impact of virtual reality applications in the treatment of anxiety disorders: a systematic review and meta-analysis of randomized-controlled trials. J Behav Ther Exp Psychiatry 2023; 81: 101893.

19.

van Loenen

Scholten

Muntingh

, et al. The effectiveness of virtual reality exposure-based cognitive behavioral therapy for severe anxiety disorders, obsessive-compulsive disorder, and posttraumatic stress disorder: meta-analysis. J Med Internet Res 2022; 24: e26736.

20.

Knowles

Tolin

. Mechanisms of action in exposure therapy. Curr Psychiatry Rep 2022; 24: 861–869.

21.

APA. What is exposure therapy? 2017. https://www.apa.org/ptsd-guideline/patients-and-families/exposure-therapy.

22.

Barlow

Farchione

Sauer-Zavala

, et al. Unified protocol for transdiagnostic treatment of emotional disorders: Workbook. New York: Oxford University Press, 2017.

23.

Carl

Stein

Levihn-Coon

, et al. Virtual reality exposure therapy for anxiety and related disorders: a meta-analysis of randomized controlled trials. J Anxiety Disord 2019; 61: 27–36.

24.

Foa

Zandberg

McLean

, et al. The efficacy of 90-minute versus 60-minute sessions of prolonged exposure for posttraumatic stress disorder: design of a randomized controlled trial in active duty military personnel. Psychol Trauma Theory Res Pract Policy 2019; 11: 307.

25.

Michelle

TQY

Jarzabek

Wadhwa

. CBT assistant: mHealth app for psychotherapy. In: 2014 IEEE global humanitarian technology conference-South Asia Satellite (GHTC-SAS), Trivandrum, India, 26 September–27 September 2014. pp.135–140. New York: IEEE.

26.

Hunt

Miguez

Dukas

, et al. Efficacy of zemedy, a mobile digital therapeutic for the self-management of irritable bowel syndrome: crossover randomized controlled trial. JMIR mHealth uHealth 2021; 9: e26152.

27.

Pramana

Parmanto

Kendall

, et al. The smartcat: an m-Health platform for ecological momentary intervention in child anxiety treatment. Telemed e-Health 2014; 20: 419–427.

28.

Newton

Bagnell

Rosychuk

, et al. A mobile phone-based app for use during cognitive behavioral therapy for adolescents with anxiety (MindClimb): user-centered design and usability study. JMIR mHealth uHealth 2020; 8: e18439.

29.

Reger

Hoffman

Riggs

, et al. The “PE coach” smartphone application: an innovative approach to improving implementation, fidelity, and homework adherence during prolonged exposure. Psychol Serv 2013; 10: 342.

30.

Brasileiro

FNV

Taunay

TCD

de Vasconcelos Filho

, et al. Exphobia: a mHealth technology based on augmented reality for support the treatment of arachnophobic stimulus. In: Proceedings of the 12th international joint conference on biomedical engineering systems and technologies (BIOSTEC 2019) – HEALTHINF, Prague, Czech Republic, 22 February–24 February 2019, pp.474–480. Setúbal, Portugal: SciTePress.

31.

Vinci

Brandon

Kleinjan

, et al. Augmented reality for smoking cessation: development and usability study. JMIR mHealth uHealth 2020; 8: e21643.

32.

Denecke

Schmid

Nüssli

, et al. Implementation of cognitive behavioral therapy in e-mental health apps: literature review. J Med Internet Res 2022; 24: e27791.

33.

Stoyanov

Hides

Kavanagh

, et al. Mobile app rating scale: a new tool for assessing the quality of health mobile apps. JMIR mHealth uHealth 2015; 3: e3422.

34.

Jessup

Putrik

Buchbinder

, et al. Identifying alternative models of healthcare service delivery to inform health system improvement: scoping review of systematic reviews. BMJ Open 2020; 10: e036112.

35.

Klasnja

Consolvo

Pratt

. How to evaluate technologies for health behavior change in HCI research. In: Proceedings of the SIGCHI conference on human factors in computing systems,Vancouver, BC, Canada, 7 May–12 May 2011, pp.3063–3072. New York: ACM.

36.

Henry

. Process-user interface alignment: new value from a new level of alignment. Align J 2007.

37.

Flood

Gleeson

Flynn

, et al. Using methods from human-centered design in health research: an introduction for pharmacy and health services researchers. In: Desselle SP, Cardenas VG, Anderson C, (eds) Contemporary research methods in pharmacy and health services. London: Elsevier, 2022, pp.103–113.

38.

Nathan

Gorman

. A guide to treatments that work. New York: Oxford University Press, 2015.

39.

Abramowitz

Deacon

Whiteside

. Exposure therapy for anxiety: principles and practice. Guilford Publications, 2019.

40.

Tindall

Ferris

Townsend

, et al. A first-hand experience of co-design in mental health service design: opportunities, challenges, and lessons. Int J Ment Health Nurs 2021; 30: 1693–1702.

41.

Figma. The collaborative interface design tool, 2022. https://www.figma.com/.

42.

Wynne

Hellesoy

Tooke

. The cucumber book: behaviour-driven development for testers and developers. Raleigh, North Carolina: Pragmatic Bookshelf, 2017.

43.

Thabrew

Fleming

Hetrick

, et al. Co-design of ehealth interventions with children and young people. Front Psychiatry 2018; 9: 481.

44.

TypeScript. Javascript with syntax for types, 2022. https://www.typescriptlang.org/.

45.

Angular, 2022. https://angular.io/.

46.

NativeScript, 2022. https://nativescript.org/.

47.

Bähr

Haas

Keusch

, et al. Missing data and other measurement quality issues in mobile geolocation sensor data. Soc Sci Comput Rev 2022; 40: 212–235.

48.

González-Pérez

Matey-Sanz

Granell

, et al. Using mobile devices as scientific measurement instruments: reliable android task scheduling. Pervasive Mob Comput 2022; 81: 101550.

49.

González-Pérez

Matey-Sanz

Granell

, et al. Awarns: a framework for developing context-aware reactive mobile applications for health and mental health. J Biomed Inform 2023; 141: 104359.

50.

Eccles

Arsal

. The think aloud method: what is it and how do I use it? Qual Res Sport Exercise Health 2017; 9: 514–531.

51.

Barlow

. Anxiety and its disorders: the nature and treatment of anxiety and panic. New York: Guilford Press, 2004.

52.

Terhorst

Philippi

Sander

, et al. Validation of the mobile application rating scale (MARS). PLoS ONE 2020; 15: e0241480.

53.

R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing, 2023. https://www.R-project.org/ .

54.

Allaire

Teague

Scheidegger

, et al. Quarto, 2024. DOI: 10.5281/zenodo.5960048. https://github.com/quarto-dev/quarto-cli.

55.

Posit team. RStudio: integrated development environment for R. Boston, MA: Posit Software, PBC, 2024. http://www.posit.co/ .

56.

González-Pérez

Matey-Sanz

. SyMptOMS mobile app for exposure therapy—SyMptOMS-ET, 2022. DOI: 10.5281/zenodo.7483861.

57.

González-Pérez

Granell

. A fully instrumentalised geolocation-and smartphone-based exposure therapy for anxiety disorders: SyMptOMS-ET—reproducibility package, 2022. DOI: 10.5281/zenodo.7483787.

58.

Lemon

Huckvale

Carswell

, et al. A narrative review of methods for applying user experience in the design and assessment of mental health smartphone interventions. Int J Technol Assess Health Care 2020; 36: 64–70.

59.

do Nascimento

Rodrigues

Rotta

, et al. Evaluation of mobile applications focused on the care of patients with anxiety disorders: a systematic review in app stores in brazil. Int J Med Inform 2023; 175: 105087.

60.

Salazar

de Sola

Failde

, et al. Measuring the quality of mobile apps for the management of pain: systematic search and evaluation using the mobile app rating scale. JMIR mHealth uHealth 2018; 6: e10718.

61.

Wei

Zheng

Deng

, et al. Design features for improving mobile health intervention user engagement: systematic review and thematic analysis. J Med Internet Res 2020; 22: e21687.

62.

Auroy

Sarradon-Eck

, et al. Patients’ perceptions of mHealth apps: meta-ethnographic review of qualitative studies. JMIR mHealth uHealth 2019; 7: e13817.

63.

Carrouel

de Vigneulles

Bourgeois

, et al. Mental health mobile apps in the french app store: assessment study of functionality and quality. JMIR mHealth uHealth 2022; 10: e41282.

64.

Rivera-Romero

Gabarron

Ropero

, et al. Designing personalised mHealth solutions: an overview. J Biomed Inform 2023; 146: 104500.

65.

Woods

Duff

Roehrer

, et al. Patients’ experiences of using a consumer mHealth app for self-management of heart failure: mixed-methods study. JMIR Hum Factors 2019; 6: e13009.

66.

Strodl

Shakespeare-Finch

Alichniewicz

, et al. Clinicians’ perceptions of PTSD coach Australia. Internet Interv 2020; 21: 100333.

67.

Agher

Sedki

Despres

, et al. Encouraging behavior changes and preventing cardiovascular diseases using the prevent connect mobile health app: conception and evaluation of app quality. J Med Internet Res 2022; 24: e25384.

68.

Knitza

Tascilar

Messner

, et al. German mobile apps in rheumatology: review and analysis using the mobile application rating scale (MARS). JMIR mHealth uHealth 2019; 7: e14991.

69.

Noorbergen

Adam

Teubner

, et al. Using co-design in mobile health system development: a qualitative study with experts in co-design and mobile health system development. JMIR mHealth uHealth 2021; 9: e27896.

70.

Bardram

. Evaluating personalized pervasive health technology—But How? IEEE Pervasive Comput 2020; 19: 37–44.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

18.02 MB

0.00 MB