Sage Journals: Discover world-class research

Abstract

Background

Real-time digital mental health interventions, primarily enabled by smartphone technology offer continuous, personalised support, that adapts in response to the changing needs of individuals. Despite being prominently explored in populations with psychiatric disorders, there remains a notable gap in the systematic analysis of demographic characteristics, as well as the foundational design decisions or rules that underpin the personalisation of these interventions.

Objectives

(a) Identifying the prevalent design decisions to enable personalisation within real-time digital mental health interventions, (b) the influence of these design decisions on the clinical outcomes of the interventions, and (c) the demographic characteristics of populations with psychiatric disorders targeted by real-time digital health interventions.

Methods

Following PRISMA (preferred reporting items for systematic reviews and meta-analyses) guidelines, a systematic literature review was conducted of peer-reviewed literature focusing on real-time digital interventions in populations with clinically diagnosed psychiatric disorders. We undertook a narrative synthesis to derive the demographics and personalisation design decisions of the interventions and conducted a pooled meta-analysis to evaluate clinical outcomes.

Results

Interventions predominantly targeted female and Caucasian demographics, yielding modest clinical improvements. Our analysis identified nine critical personalisation design decisions concerning measurement, intervention, and interactions with health professional with varying influence on clinical outcomes.

Conclusion

Understanding the complex nuances of design decisions that shape real-time digital health interventions, as well as identifying which patient demographics benefit most, is fundamental for their effective clinical impact and safe use.

Prospero Registration

PROSPERO CRD42020161663.

Keywords

Mobile health ecological momentary interventions tailoring personalisation mental health digital health interventions digital mental health interventions real-time mHealth

Introduction

Globally, one in eight people experience mental illness at some point in their life, contributing to approximately one fifth of all years lived with a disability.¹ During the COVID-19 pandemic, in particular, the already under-resourced global mental health infrastructure contracted significantly, with 93% of all countries experiencing a disruption or halting of critical mental health services.² As demand for mental health treatment continues to exceed the capacity of existing healthcare services,³ digitally-enabled healthcare delivery appears to be a viable cost-effective solution to improve timely and equitable access to care.⁴

Digital mental health interventions (DMHIs) emerged in the early 2000s as an extension of traditional therapies, but delivered over the internet.^5–7 With the advent of smartphones and wearables in the early 2010s, the ability to use objective indicators along with subjective ecological momentary assessments (EMAs) as proxy markers of clinical symptoms and mental states made it possible to offer real-time feedback and adjust interventions dynamically.^8–11 Unlike internet-based platforms or traditional websites, real-time interventions are used more frequently during day-to-day life for specific tasks, presenting more targeted and personalised “in-the-moment” behaviour change support related to mental health.¹²

The role of mobile health (mHealth) solutions in managing health, wellbeing, and chronic illness has been explored and evaluated across many aspects of health, including nutrition,¹³ physical activity,¹⁴ and disease monitoring.¹⁵ In mental healthcare, the feasibility, effectiveness, and adoption of mHealth have become increasingly evident, particularly for conditions like depression, anxiety, and post-traumatic stress disorder.^16–18 Moreover, widespread access and the near ubiquitous use of smartphone¹⁹ uniquely positions mHealth to provide “personal, intelligent, [and] connected”²⁰ mental health interventions at scale.^21,22

Real-time DMHIs also known as ecological momentary interventions (EMIs)²³ or just-in-time adaptive interventions (JITAIs)²⁴ are terms for a category of mHealth interventions that use data gathered during day-to-day life to tailor and deliver interventional content in real time. Real-time DMHIs follow a pre-established set of rules or decision-points that determine what interventional content should be delivered and when it should be delivered i.e., the timing and frequency of delivery. This determination is refined by incorporating feedback data gathered after content delivery, thereby tailoring the rules themselves to meet the individual needs of each user.

Real-time DMHIs targeting individuals with mental illness offer considerable potential to advance personalised healthcare. For instance, real-time DMHIs tailored for depression and anxiety have shown positive results treating common psychiatric conditions, such as depressive and general anxiety symptoms, stress, and social anxiety. The efficacy of smartphone-based real-time DMHI studies has been well-established across several randomised controlled trials and confirmed by meta-analyses.^17,25 Despite this, low adoption and poor engagement with treatment are often cited as obstacles to the effectiveness of real-time DMHIs in real-world clinical practice settings.²⁶ To address the issues of poor adoption and to achieve more effective clinical outcomes, Lipschitz et al.²⁷ suggest that real-time DMHIs must engage users more reliably and consistently. Although recent reviews have explored the effectiveness of real-time DMHIs, they have done so narrowly in relation to specific psychiatric conditions,^28–30 or with an emphasis on the purpose of the technology,³¹ or a specific feature of the intervention.^32,33 Given the complexity of real-time DMHIs, the extent to which design aspects of real-time DMHIs influence engagement and impact clinical outcomes has yet to be analysed.

Existing theoretical frameworks provide valuable insights into the possible design, development, and implementation of DMHIs; however, they do not consider the real-time dynamic feedback mechanisms characteristic of real-time DMHIs.³⁴ While foundational models like the Health Belief Model³⁵ and Social Cognitive Theory³⁶ incorporate general user health behaviours and attitudes, they often lack specificity regarding how different demographic factors, such as age, gender, and socioeconomic characteristics, influence engagement and outcomes. Furthermore, existing models do not fully account for ecological validity and contextual factors that affect the use of DMHIs in clinical populations.⁸ Moreover, theoretical frameworks often emphasise initial adoption and short-term outcomes but do not adequately address long-term engagement and the sustainability of digital interventions, nor their clinical effectiveness.

Accordingly, comprehensive systematic reviews of the design aspects of real-time DMHIs and the populations of individuals with mental illness in which they were implemented are needed to inform translation and future research directions. We systematically reviewed real-time DMHI research studies to understand and document the design decisions of the interventions employed and the demographic characteristics of the treated populations, while analysing clinical outcomes [PROSPERO protocol number: CRD42020161663].

The primary research questions guiding this systematic review are: What are the critical design decisions utilised for personalisation within real-time DMHIs, and what impact do they have on the clinical outcomes of these real-time DMHIs? Additionally, this research addresses a secondary question to provide a comprehensive understanding of the topic. Specifically, it examines the different user demographic characteristics, such as age, gender, and socioeconomic factors, targeted by real-time DMHIs. By addressing these questions, the study aims to identify key factors that influence the effectiveness of DMHIs in populations with clinically diagnosed psychiatric disorders.

Methods

This review conforms with and is reported according to the preferred reporting items for systematic review and meta-analysis (PRISMA) 2020 guidelines³⁷ (Supplementary files 1 and 2).

Search strategy

Systematic searches of OVID, Medline, IEEE, ACM, Scopus, and PsycINFO databases were performed on 3 December 2019, and again on 15 July 2021 for studies published from 2009 onwards (in line with the emergence of research publications utilising smartphone technology). The PsycINFO search strategy is shown in Table 1, with further search strategies outlined in Appendix A.

Table 1.

PsycINFO search strategy

Construct of Interest	Search Terms
Population	addict.mp OR agoraphob.mp OR “alcohol abus”.mp OR “alcohol dependen”.mp OR “alcohol misus”.mp OR anorexi.mp OR anxi.mp OR automutilat.mp OR “binge eat”.mp OR bipolar.mp OR bulimi.mp OR depress.mp OR “eating disorder”.mp OR “general anxi”.mp OR “generalised anxi”.mp OR BPD.mp OR “impuls* disord”.mp OR insomnia.mp OR mani.mp OR “mental disorder”.mp OR “mental health”.mp OR “mental illness”.mp OR mood.mp OR neuro.mp OR “obsess compuls”.mp OR panic.mp OR phobi.mp OR psycho.mp OR “psychologic* health”.mp OR “borderline personality disorder”.mp OR schizo.mp OR “self harm”.mp OR “self injur”.mp OR “sleep disorder”.mp OR “sleep disturbance”.mp OR “sleep problem”.mp OR “social anxi”.mp OR stress.mp OR “substance abus”.mp OR “substance misus”.mp OR suicid.mp OR trauma.mp
Intervention	intervention.mp OR feedback.mp OR randomi?ed control.mp OR RCT.mp
Comparator	ANY
Outcomes	"ambulatory method”.mp OR “ecological momentary assessment”.mp OR “ecological momentary itervent”.mp OR “electron diar”.mp OR EMA.mp OR EMI.mp OR ESM.mp OR “experien monit”.mp OR “experien self monitoring”.mp OR “experien* sampl”.mp OR “self monitor”.mp
Exclude	not (review or protocol or “meta-analysis”).ti BOOKS

Study selection criteria and process

The full inclusion and exclusion criteria for study selection are shown in Table 2. We excluded studies focused on smoking cessation due to insufficient reporting of Tobacco Use Disorder (TUD) diagnoses. Instead, to ensure adequate representation of addiction disorders, we included clinically diagnosed populations with Substance Use Disorder (SUD). Notably, we included studies that reported clinical outcomes for interventions targeting clinical psychiatric populations and those incorporating feedback mechanisms into the treatment. However, studies lacking a pre and post intervention clinical outcome comparison or not classified as intervention studies were omitted. Additionally, we considered only interventions lasting a week or longer (>= 7 days) to account for the longitudinal clinical impact of the adaptive content of real-time DMHIs, in line with clinical practice standards.

Table 2.

Inclusion and exclusion criteria for studies.

Inclusion Criteria	Exclusion Criteria
(1) Published in English;	(1) were a protocol, meta-analysis or review paper;
(2) included a clinical psychiatric population — defined by:
a) above a clinical cut-off on a self-report scale,
b) diagnosed in a structured clinical interview, or
c) if participant eligibility was determined by diagnosis provided by medical practitioners;	(2) focussed on clinical populations with autism spectrum disorder, attention deficit hyperactivity disorder, sleep-wake disorders, or other neuro-developmental disorders as defined by the DSM-5 criteria;
(3) included a clinical (e.g., not engagement-based) intervention based on either self-reported or sensor data gathered during day-to-day life;	(3) entailed smoking cessation in non-clinical populations;
(4) had some interventional content significantly changed as a result of either self-reported or sensor data gathered collected during day-to-day life;	(4) did not focus on a clinically diagnosed population; and
(5) included a clinical outcome measure (i.e., not just acceptability/user satisfaction or feasibility	(5) had a sample size of ≤ 5.
(6) had an intervention of ≥ 7 days.

Titles and/or abstracts of retrieved studies were screened by two authors (WD, DT) to identify research that potentially satisfied the inclusion criteria. The full text articles of potentially eligible studies based on title and abstract review were retrieved and independently assessed for eligibility by two authors (WD, DT). Discrepancies in perceived article eligibility were resolved by a third reviewer (NB).

Data extraction and analysis

Using Covidence's Data Extraction 2.0 feature, the following details were extracted from eligible articles: First Author, Year, Location, Study Duration, Study Title, Study Design, Outcome Measures, and a brief description of the tailored intervention. For identification of real-time DMHI design decisions, the introduction, background, and methods text from each article was imported into NVivo for coding. Using an inductive approach to thematic analysis, initially, three studies were analysed by DT and WD to determine a preliminary collection of descriptive codes. Discrepancies were discussed and resolved between DT, WD and NB to agree on a final set of codes categorised under themes. Once consensus was reached DT then coded the remaining studies to reveal all the distinct design decisions.

Statistical analysis

For our analysis, the individual arms of all included studies were identified as either treatment-as-usual (TAU) arms or real-time DMHI arms. Arms were considered ‘EMI’ if the intervention delivered was adapted based on the real-time data gathered. The primary clinical outcome for each study was identified. The pre- and post-intervention mean and standard deviation scores of the arms were extracted and converted to standardised mean distance method to enable pooled comparison.³⁸ A meta-analysis was performed using both fixed and random effects models, via the function ‘metacont’ of the ‘meta’ package in R. A table of the pre-post effect sizes (Cohen's d) was then generated with a forest plot with just the intervention arms. Effect sizes were reported with standard deviations (SD) for consistency. Heterogeneity was assessed using key statistics from the random effects model and a Cochrane's Q-test: Chi-squared (χ^2), Tau-squared (τ^2) and I-squared (I^2).

Quality assessment

We report the results of a quality assessment of each included study — for which we used the quantitative randomised and non-randomised controlled trials checklist questions of the Mixed Methods Appraisal Tool (MMAT), a critical appraisal tool for systematic reviews.³⁹ All studies were appraised independently by two reviewers, DT and WD. Discrepancies between assessments were discussed and resolved via consensus between the two authors. The MMAT criteria was generated for all the included studies in Table 5.

Results

Overview

A total of 13 studies met the eligibility criteria of this review and were included in the meta-analyses (see Figure 1 and Table 3). The included studies comprised 24 arms in total, seven control (CTL) arms and 17 ‘EMI’ arms (see Table 4). Of the CTL arms, three were treatment as usual arms (TAU), two were waitlist control arms, and two were attention-control arms. Quality assessments (Table 5) noted concerns regarding appropriate representative sampling of participants in some studies,^40–43 insufficiently accounting for confounders in the design and analysis of two studies^42,43 and adherence concerns in three studies.^44–46

Figure 1.

PRISMA flowchart.

Table 3.

Study characteristics.

First Author, Year, Location	Title	Study Design	Outcome Measures*	Intervention Length	Interventions
Aardoom et al., 2016; Netherlands31	Web-Based Fully Automated Self-Help With Different Levels of Therapist Support for Individuals With Eating Disorder Symptoms: A Randomized Controlled Trial	4-arm RCT	SEED; EDE-Q; ED-QOL; PHQ-4; PTQ; TiC-P	8 weeks	Feedback Messages based on state
Bell et al., 2019; Australia32	Pilot randomised controlled trial of a brief coping-focused intervention for hearing voices blended with smartphone-based ecological momentary assessment and intervention (SAVVy): Feasibility, acceptability and preliminary clinical outcomes	Single-blind, parallel group pilot RCT	Feasibility; acceptability (CEQ; WAI-SR); Clinical outcome – PSYRATS-AH; DASS-2×; SEPS	8 weeks	Co-generated coping skills
Bell et al., 2019; Australia32		Single-blind, parallel group pilot RCT		8 weeks	Feedback of data and tailoring in therapy session
Blevins et al., 2020; USA33	Project CHOICE: Choosing healthy options in coping with emotions, an EMA/EMI plus in-person intervention for alcohol use	Pilot cohort study	Feasibility and acceptability, MDMQ-R (alcohol use motives), RAPI (problematic drinking), TLFB (quantifies alcohol use)	6 weeks	Co-generated coping skills for avoiding drinking
Burton et al., 2016; Romania, Spain, Scotland, UK34	Pilot randomised controlled trial of Help4Mood, an embodied virtual agent-based system to support treatment of depression	Pilot RCT with 2 parallel groups	Feasibility and acceptability, Depression: BDI-2; QIDS-SR; DAS-SF2; EQ-5D-5L	4 weeks	Affect of virtual agent
Burton et al., 2016; Romania, Spain, Scotland, UK34		Pilot RCT with 2 parallel groups		4 weeks	Suicide safety threshold
Depp et al., 2015; USA35	Augmenting Psychoeducation with a Mobile Intervention for Bipolar Disorder: A Randomized Controlled Trial	Single-blind RCT	MADRS; YMRS; IIS (self-report)	12 weeks	Strategies for managing warning signs
Depp et al., 2019; USA36	Single-Session Mobile-Augmented Intervention in Serious Mental Illness: A Three-Arm Randomized Controlled Trial	Three-arm RCT	Feasibility and accessibility; BPRS; MCCB; SLOF (functioning); DPAS (defeatist attitudes); DSMB (A/E)	12 weeks	Mood/Voices Intervention
					Socialisation intervention
					Med. Adherence intervention
Dulin et al., 2014; USA37	Results of a Pilot Test of a Self-Administered Smartphone-Based Treatment System for Alcohol Use Disorders: Usability and Early Outcomes	Single-arm, pre-post trial	TLFB (drinking quant.), Usability	6 weeks	GPS triggered location change prompts
					Cravings management intervention
					Feedback of EMA data
Hanssen et al., 2020; Netherlands38	An ecological momentary intervention incorporating personalised feedback to improve symptoms and social functioning in schizophrenia spectrum disorders	Two-arm, single-blinded RCT	PANSS; CAPE; SFS; WAIS-III; Feasibility	3 weeks	Feedback of data to participants
					Behaviour change prompts
					Drug abstention intervention
Moitra et al., 2021; USA39	Development and Initial Testing of an mHealth Transitions of Care Intervention for Adults with Schizophrenia-Spectrum Disorders Immediately Following a Psychiatric Hospitalization	Single-arm pre-post trial	Feasibility and acceptability: CSQ-8; SUS; USE ;AMBAS; BARS; brief COPE, BPRS; WHODAS	4 weeks	Symptom tracking/gmt. intervention
					Med. Adherence intervention
					Feedback of data from the past week
Prada et al., 2017; Switzerland40	EMOTEO: A Smartphone Application for Monitoring and Reducing Aversive Tension in Borderline Personality Disorder Patients, a Pilot Study	Pilot study	Feasibility and usability; Aversive tension; BDI at baseline	24 weeks	Distraction exercises for different states of aversive tension
Proudfoot et al., 2013; Australia41	Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial	Three-arm RCT	DASS-2×; WSAS	7 weeks	Tailored psychoeducation modules
Proudfoot et al., 2013; Australia41		Three-arm RCT	DASS-2×; WSAS	7 weeks	User-determined self-monitoring
Shapiro et al., 2010; USA42	Mobile Therapy: Use of Text-Messaging in the Treatment of Bulima Nervosa	Single-arm pre-post trial	BDI; EDI; BPQ; NEQ; acceptability	12 weeks	Text-message based goal-achievement feedback for behaviour abstinence
Wenze et al., 2016; USA43	An Open Trial of a Smartphone-Assisted, Adjunctive Intervention to Improve Treatment Adherence in Bipolar Disorder	Single-arm pre-post trial	QIDS-C; CARS-M; MSSI; MDIS; BMQ; WAI; MHLC; CES; MCQ; TAF	12 weeks	Feedback messages with tailored coping strategies
Wenze et al., 2016; USA43		Single-arm pre-post trial	QIDS-C; CARS-M; MSSI; MDIS; BMQ; WAI; MHLC; CES; MCQ; TAF	12 weeks	Morning and evening suicide risk assessment and feedback

Table 4.

Study arms across 13 included studies.

Study	Population	Length of Intervention	Scale	Baseline		Outcome		User Triggers Measurement	Delivery prompting frequency	Measurement prompting frequency	One size fits all vs person specific content	Content refreshed over time	Was there a therapist	Did therapist have access and use data	Real time sensor Measurement	Prompt triggered by patterns detected in accumulated data
	(n)	(weeks)		Mean	SD	Mean	SD						(yes/no)	(yes/no)
Aardoom 2016-EMI	87	8	EDE-Q	4.24	0.8	4.05	1	No user trigger	>1/day	< weekly	One size fits all	No	No	No	No	No
Aardoom 2016-EMI	88	8	EDE-Q	4.43	0.94	3.81	1.18	No user trigger	>1/day	< weekly	One size fits all	No	Yes	No	No	No
Aardoom 2016-EMI	89	8	EDE-Q	3.95	0.79	3.58	1.32	No user trigger	>1/day	< weekly	One size fits all	No	Yes	No	No	No
Aardoom 2016-TAU	90	8	EDE-Q	4.05	1.11	3.92	1.14	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Bell 2019-BMI	17	8	PSYRATS- AH	28.47	4.87	25.89	6.37	No user trigger	1/day-2/week	1/day-2/week	Person specific content	Yes	Yes	Yes	No	No
Bell 2019-TAU	17	8	PSYRATS- AH	28.76	4.41	29.47	6.45	N/A	N/A	N/A	N/A	N/A	Yes	Yes	N/A	N/A
Blevins 2020- EMI	20	6	RAPI	13.23	10.83	4.31	4.5	No user trigger	>1/day	>1/day	Person specific content	No	No	No	No	Yes
Burton 2016-EMI	14	4	BDI-2	19.6	8.1	13.9	8.1	No user trigger	>1/day	1/day-2/week	One size fits all	No	Yes	Yes	Yes	No
Burton 2016-TAU	14	4	BDI-2	21.8	6.8	17.6	6.8	N/A	N/A	N/A	N/A	N/A	Yes	Yes	N/A	N/A
Depp 2015-EMI	41	12	YMRS	7.4	6	5.7	5.8	No user trigger	>1/day	>1/day	Person specific content	No	Yes	Yes	No	Yes
Depp 2015-EMI	41	12	YMRS	4.8	4.7	5	4.2	User triggered	1/day-2/week	1/day-2/week		No	Yes	Yes	No	Yes
Depp 2019-EMI	77	12	BPRS-24	43.2	9.7	39.7	11.5	No user trigger	>1/day	>1/day	Person specific content	No	Yes	Yes	No	Yes
Depp 2019-EMI	69	12	BPRS-24	41.6	11.5	38.6	9.1	No user trigger	N/A	>1/day	N/A	N/A	Yes	Yes	N/A	N/A
Depp 2019- TAU	83	12	BPRS-24	42	10.9	41	11.4	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Dulin 2014- EMI	28	6	Avg drink per day	5.6	2.9	2.9	2.4	User triggered	1/day-2/week	1/day-2/week	Person specific content	Yes	No	No	Yes	No
Hanssen 2020- EMI	27	3	PANSS	23.9	7.9	22.1	7.21	No user trigger	>1/day	>1/day	One size fits all	No	No	No	No	Yes
Hanssen 2020- TAU	23	3	PANSS	24.73	8.44	23.3	7.35	No user trigger	N/A	>1/day	N/A	N/A	No	No	No	No
Moitra 2021- EMI	10	4	BPRS	46.4	6.7	30.4	8.5	User triggered	>1/day	>1/day	One size fits all	No	No	No	No	No
Prada 2017- EMI	16	24	Aversive tension	5.95	3.13	2.83	2.36	User triggered	>1/day	1/day-2/week	One size fits all	No	No	No	No	No
Proudfoot 2013- EMI	231	7	DASS-21	40.53	18.94	31.2	19.31	User triggered	1/day-2/week	1/day-2/week	One size fits all	Yes	No	No	Yes	Yes
Proudfoot 2013-TAU	246	7	DASS- 21	42.4	19.53	39.01	23.94	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Proudfoot 2013- TAU	228	7	DASS- 21	42.6	18.45	41.56	22.11	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A	N/A
Shapiro 2010-EMI	31	12	EDI-2	23.1	10.7	11.4	9.6	No user trigger	>1/day	1/day-2/week	One size fits all	No	No	No	No	No
Wenze 2016-EMI	8	12	MDIS	11.4	0.82	11.54	0.58	No user trigger	1/day-2/week	>1/day	One size fits all	No	No	No	No	Yes

Note: EMI = Ecological Momentary Intervention: TAU = Treatment as Usual; EDE-Q = Eating Disorder Examination Questionnaire; PSYRATS-AH = Psychotic Symptom rating Scales; RAPI = Rutgers Alcohol problem index; BDI-2 + Beck Depression Inventory scale; YMRS = Young Mania Rating Scale; BPRS-24 = Brief Psychiatric Rating Scale; PANSS = Positive and Negative Syndrome Scale; DASS-21 = Depression Anxiety Stress Scale; EDI = Eating Disorder Inventory Scale; MDIS = Mood Disorder Insights Scale.

Table 5.

Quality assessment of included studies using the mixed methods appraisal tool (MMAT).

	Aardoom et al., 2016	Bell et al., 2019	Blevins et al., 2020	Burton et al., 2016	Depp et al., 2015	Depp et al., 2019	Dulin et al., 2014	Hanssen et al., 2020	Moitra et al., 2021	Prada et al., 2017	Proudfoot et al., 2013	Shapiro et al., 2010	Wenze et al., 2016
Study Design*	RCT	RCT	NRQ	RCT	RCT	RCT	NRQ	RCT	NRQ	NRQ	RCT	NRQ	NRQ
Is randomization appropriately performed?	Yes	Yes		Yes	Yes	Yes					Yes
or, are participants representative of the target population?			Can't tell				No	No	Yes	No		Yes	No
Are the groups comparable at baseline	No	Yes		Yes	Yes	Yes					Yes
or, are measurements appropriate regarding both the outcome and intervention (or exposure)			Yes				Yes	Yes	Yes	No		Yes	Yes
Are there Complete outcome data	No	Yes		Yes	Yes	Yes	Yes	Yes			No	Yes	Yes
Are the outcome assessors blinded to the intervention provided?	Yes	Yes		No	Yes	Yes					No
or, Are the confounders accounted for in the design and analysis?			Yes				Yes	Yes	Yes	No		Yes	No
Did the particip[ants adhere to the assigned intervention?	No	Yes			No	No					No
or, During the study period, is the intervention administered (or exposure occurred) as intended			Yes				Yes	Yes	Yes	Yes		No	Yes

Population characteristics

Across all 13 studies, there were 1595 participants in total; ranging from 8 participants⁴³ to 705 participants⁴⁶ per study. Detailed demographic characteristics are presented in Appendix B. The mean age of the participant pool is 36.62 years (9.926 years SD), and participants were predominantly female (1135/1595; 71.16%) with 28.77% (459/1595) males. Eight out of the 13 studies reported participants’ ethnicities according to their respective countries’ conventions.

Participants’ primary psychiatric diagnoses for the purposes of inclusion in the studies were able to be grouped into six broad diagnostic groups; in order of largest to smallest number of participants: borderline personality disorder (BPD; 721/1595; 45.20%), eating disorders (ED; 385/1595; 24.14%), schizophrenia-spectrum disorders (SSD; 260/1595; 16.30%), bipolar affective disorders (BPAD; 152/1595; 9.530%), substance use disorders (SUD; 48/1595; 3.009%), and major depressive disorder (MDD; 29/1595; 1.818%). The SSD group included primary diagnoses of schizophrenia, schizoaffective disorders, schizophreniform disorders, and psychotic disorders. The ED group included participants with anorexia nervosa and bulimia nervosa. Both studies^40,47 classified under the SUD group primarily focused on alcohol use.

Outcomes

The standardised pre-post mean differences of the 17 ‘EMI’ arms in the 13 included studies are presented as forest plot in Figure 2. Most studies showed a reduction in illness-related psychopathology. One study⁴³ outcome measure was positively coded (i.e., positive outcome scores indicated an improvement in state-psychopathology); accordingly, we negatively recoded this outcome measure to reflect the net reduction in psychopathology and enable comparison with other studies. Reported data describing periods for relapse or, for substance use disorders (SUDs), duration of abstinence, were not included in the analysis due to a lack of consistent and comparable reporting standards.

Figure 2.

Forest plot.

Heterogeneity test results yielded a Chi-squared value of 35.64, df = 16 (P < 0.01). This indicates that there is significant heterogeneity among the included studies. However, the estimated Tau-squared value was 0.0583, indicating a small amount of between-study variance beyond sampling error. Additionally, the I-squared value of 55% suggests moderate heterogeneity among the included studies, implying that approximately 55% of the total variation in effect sizes can be attributed to true differences between studies.

Design decisions and outcomes

As a result of the thematic analysis, we identified nine distinct design decisions across three themes within the 17 included ‘EMI’ arms. Table 6 lists these design decisions within their associated themes, alongside mean pre-post outcome effect-sizes (Cohen's d) associated with each design factor.

Table 6.

Effect sizes of the identified EMI characteristics (EMI = ecological momentary intervention; n = number of arms with the EMI characteristic).

Design Characteristics	N	Effect Size	Standard Error of Mean	Standard Deviation
MEASUREMENT DESIGN
1. How often were participants assessed?
More than one time per day	8	−0.59	0.27	0.76
Once per day → weekly	7	−0.67	0.15	0.39
Less than weekly	3	−0.46	0.12	0.21
2. Did the user trigger the assessment?
Non-user triggered	13	−0.46	0.08	0.29
User triggered	5	−0.95	0.4	0.9
3. Were real-time sensor measures incorporated into the design?
N/A	1	−0.26
No	14	−0.6	0.16	0.61
Yes	3	−0.71	0.13	0.22
INTERVENTION DESIGN
4. How often were participants offered an intervention?
N/A	2	−0.22	0.05	0.06
More than once a day	11	−0.75	0.19	0.62
Once a day to a couple of times weekly	5	−0.42	0.17	0.37
5. Were interventions triggered by patterns detected in accumulated data, or according to a pre-set schedule?
N/A	1	−0.26
Schedule-based	10	−0.82	0.2	0.63
Pattern-based	7	−0.33	0.1	0.27
6. Was intervention content assigned on a participant-by-participant basis, or was the intervention content the same for all participants?
N/A	2	−0.22	0.05	0.06
One-size fits all	10	−0.74	0.21	0.66
User-specific content	5	−0.59	0.13	0.28
7. Was the intervention content refreshed over the course of the intervention, or did it remain the same from beginning to end?
N/A	2	−0.22	0.05	0.06
Unchanged	13	−0.64	0.17	0.62
Refreshed	3	−0.65	0.14	0.24
HEALTH PROFESSIONAL INTERFACE
8. Was there a therapist/clinician involved?
No	10	−0.75	0.22	0.68
Yes	8	−0.4	0.09	0.24
9. Did the therapist/clinician have access to and use data collected by the intervention?
No	12	−0.72	0.18	0.62
Yes	6	−0.35	0.1	0.25

Discussion

Population demographics

The population demographics observed in the included real-time DMHI studies, where reported, primarily featured female and Caucasian participants with an evident underrepresentation of individuals with lower education levels and those who are unemployed. Accordingly, the real-time DMHI research literature reviewed here has limited generalisability beyond the sampled populations. Social determinants, such as employment status,⁴⁸ level of education⁴⁹ and digital health literacy⁵⁰ emerged as pivotal factors influencing study recruitment bias, treatment response and adherence. To address this, targeted recruitment strategies may be adopted in future to resolve gender-bias and ensure broader representation across socio-demographic groups, including cultural and linguistically diverse (CALD) populations. Additionally, user-friendly interfaces that accommodate varying levels of digital ability, language translations, and culturally tailored content are essential future design considerations to enhance the accessibility and effectiveness of real-time DMHIs across diverse populations.

Of the six main clinical mental illness diagnostic groups treated with real-time DMHIs, participants with BPD were the largest group, followed by the ED group. This real-time DMHI research representation is potentially disproportionate to these conditions’ respective diagnostic mental illness prevalence in the general population; for example, highly prevalent MDD affects up to 3.8% of the general population.⁵¹ There is a need for further primary evidence exploring the effect of these interventions in MDD. Although many studies did focus on depressive symptoms or states; this was in the context of other clinical diagnoses rather than MDD itself.

Design decisions

Intervention periods ranged from 3–24 weeks across studies with the most common intervention length being twelve weeks. The frequency of symptom measurements (how often were participants assessed?) showed little difference across studies in terms of pre-post effect sizes. Similarly, the intensity of intervention delivery (how often were participants offered an intervention?) also demonstrated minimal effect size differences when comparing high intensity (‘More than once a day against’) to lower intensity (‘Once a day to a couple of times weekly’) interventions. Also, effect sizes were similar whether real-time sensor measures were incorporated in the design of the intervention or not.

Interventions incorporating a user-triggered component reported the largest pre-post outcome effect size (d = -0.95), indicating significant improvements in clinical outcomes. A user-triggered approach gives users full control, or ‘agency’, to actively engage with the intervention. Bandura³⁶ defines agency as the capacity to influence a course of events by one's actions, a critical element when tailoring DMHIs to the specific needs and contexts of individual users. The large effect size observed may be attributed to the fact that user control allows interventions to be more appropriately tailored and aligned with user's personal goals, addressing the challenge of fully understanding a user's clinical or mental state based solely on quantifiable data.⁵² Agency enabled by user-triggered personalisation enhances the intervention's relevance, ensuring it is delivered more precisely at a time of need and when it is most likely to be well-received, thereby improving clinical outcomes.

Recent theoretical advancements in behaviour change theory underscore the critical role of user agency, asserting that when individuals have control over the timing, content, and delivery; digital health interventions become significantly more relevant.⁵³ This control allows for a better understanding of the unique nuances in each person's daily life that algorithm-triggered approaches may overlook. Ultimately, the success of real-time digital health interventions - regardless of how tailored they are - depends on the extent to which users initiate them, as they are best positioned to understand their own contexts and needs. Notably, only five out of the thirteen analysed studies in this review included user-triggered functionality, which raises several possibilities for future research to explore. For example, do users trigger interventions when their symptom burden is most taxing, and they sense an opportunity for relief? Does user agency and initiative in turn encourage self-driven engagement with the intervention?

Conversely, the real-time DMHI design decision that demonstrated the lowest pre-post change was for interventions that were triggered by pattern-based detections in accumulated sensor data (d = -0.33). Given that real-time DMHIs that did not incorporate this feature, using a schedule-based approach to intervention delivery instead, had the second greatest effect size (d = -0.82), the design decision to trigger interventions based on patterns detected in accumulated data has limited benefits. There are several rationales at various levels of the intervention which could explain this result. At the level of measurement design, the information gathered could be an inappropriate reflection of the user's clinical state, whether it be unsuitable timing for that individual user, or use of an unsuitable type of measurement itself. On the intervention level, the decision-making system may have a fundamental fault that causes the delivery of an inappropriate intervention and/or at the inappropriate time.

Prior systematic reviews and meta-analyses have explored various aspects of DMHIs. For instance, a systematic review and meta-analysis by Gan et al. demonstrated that greater engagement with DMHIs is significantly associated with improved mental health outcomes, regardless of intervention type or mental health condition targeted.⁵⁴ Similarly, a meta-review by Goldberg et al. synthesised findings from 14 meta-analyses and highlighted the broad potential of mobile phone-based interventions to reduce psychological symptoms and improve quality of life.⁵⁵ However, these reviews primarily focus on the general efficacy and engagement aspects of DMHIs without investigating the specific design decisions that influence their effectiveness. In contrast, the present research systematically examines how critical design features (e.g., measurement frequency, intervention delivery intensity, how interventions are triggered, and the use of sensor data) influence the clinical effectiveness of real-time DMHIs. This focus on the design elements provides a more nuanced understanding that informs the development of effective DMHIs.

Digital real-time DMHIs in comparison to traditional intervention delivery

The total effect size across all ‘EMI’ arms showed a small improvement in clinical state or symptoms (d = -0.50 [−0.66; −0.33]; N = 894). The observed effect sizes were smaller than those of other DMHIs. Specifically, Elzelmueller et al.⁵⁶ and Newby et al.⁵⁷ found larger pre-post effects of internet-based digital health interventions on depression and anxiety, while Bear et al.⁵⁸ reported a substantially higher pre-post improvement effect of routine specialist mental health care for young people with depression and/or anxiety. Of note, some research suggests that anxiety and unipolar depression are more responsive to behavioural health treatment than personality disorders or serious mental illness.⁵⁹ Therefore, it is difficult to compare the effect sizes of real-time DMHIs in this meta-analysis to those in meta-analyses of traditional DMHIs that are weighted more heavily on unipolar depression and anxiety disorders. Additionally, the majority of the real-time DMHIs included in this meta-analysis did not include human support. Given consistent findings that digital health interventions with human interaction are more effective and engaging than those without,⁶⁰ it may be important to limit future comparisons to only those interventions without human support.

Quality of included studies

Rigorous quality assessments of the included studies indicated that they were of high quality overall. Notably, all studies met critical appraisal criteria for appropriate randomization. However, two studies failed to account for confounders in the design and analysis, and several studies reported concerns related adherence to treatment. Only two of the 13 included studies met all five critical appraisal items of the MMAT. Future research in the field may benefit from refining recruitment and sampling techniques, ensuring consistent outcome data collection, correctly addressing cofounders effectively and prioritising attention to adherence.

Study diversity

Seemingly conflicting results suggest that while there is a significant amount of heterogeneity as per the Q-test, the actual differences between the study effects (as indicated by τ^2 value of 0.0583) are relatively small, which may be attributed to the diverse nature of the interventions and outcomes measured across the small number of included studies. The moderate I^2 value of 55% supports this by showing that over half of the variability in effect sizes is due to heterogeneity rather than chance, suggesting that while there is some consistency in the effects observed across studies, there are also differences that warrant further investigation. Future research should aim to better understand the sources of heterogeneity, such as the varied aspects of intervention types, dosing, follow-up durations and treatment protocols associated with real-time DMHIs, and consider them when reporting reviews.

Limitations of the review

The diversity of clinical populations and variety of reported clinical outcomes – ranging from different quality-of-life measures to an assortment of subjective assessments of symptoms even within psychiatric conditions – presented complexities during data analysis. Differences in methodologies, follow-up periods and data collection approaches across the included studies compounded this challenge of synthesizing findings and highlights the need for cautious interpretation. Furthermore, the clinical implications of specific scores often lacked detail, with scarce indications of severity and cut-off or threshold scores for recovery and remission in the given populations. To address these complexities, consensus on validated clinical outcome measures for specific psychiatric conditions and consistent reporting standards would be beneficial.

Despite these limitations, comparing standardised mean differences remains a valuable approach to understanding the impact of real-time DMHI designs on clinical outcomes for individuals with mental illness. However, caution should be exercised in interpreting these preliminary findings too substantively considering the study diversity within this systematic review was limited to research settings. The presented findings aim to guide future systematic reviews that bring clarity to the field and to inspire further experimental research, emphasising experimental rigor and design considerations with a view to addressing the practical challenges in implementing real-time DMHIs in real-world settings.

Conclusion

Despite the variability in their designs, target populations, and evaluation metrics, real-time digital mental health interventions (DMHIs) are emerging as a promising avenue for delivering personalised mental healthcare. These interventions have the potential to address significant gaps in mental health care by providing personalized, accessible and scalable solutions. However, the overrepresentation of females and Caucasian individuals within these studies casts doubt on the applicability of these interventions to wider population. This observation warrants further research to address barriers to participation amongst under-represented demographics and to ensure that real-time DMHIs are effective across culturally and linguistically diverse (CALD) populations.

The efficacy of these interventions appears to be influenced by specific design choices that personalise the user experience. Notably, interventions that empower users to initiate data measurements appear to be more effective, suggesting that user autonomy is a critical factor in the success of this approach. Future research must prioritise design for user agency to fully harness the potential of real-time DMHIs. Additionally, the scalability of successful interventions is crucial for their wider adoption and effectiveness in real-world settings. Factors such as technological infrastructure, user training, and integration with existing healthcare systems will play a significant role in determining the scalability of these interventions.

Broader implications for mental health policy and practice include the potential for real-time DMHIs to provide continuous, real-time support, thereby reducing the burden on traditional mental health services and improving patient outcomes. Policymakers should consider integrating real-time DMHIs into standard care protocols to enhance accessibility and efficiency. Future research directions should include longitudinal studies to assess the long-term impact of these interventions, diverse population samples to ensure generalisability, and exploration of different psychiatric conditions to broaden the applicability of real-time DMHIs. Combining expertise from fields such as psychology, psychiatry, technology, healthcare, and social sciences through interdisciplinary collaboration during development and implementation will ensure that these DMHIs are both scientifically robust and practically feasible. By addressing these areas, researchers can advance the field of digital mental health and create interventions that are inclusive, effective, and sustainable.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241306782 - Supplemental material for Critical design decisions and user demographics in enhancing real-time digital mental health interventions: A systematic review

Supplemental material, sj-docx-1-dhj-10.1177_20552076241306782 for Critical design decisions and user demographics in enhancing real-time digital mental health interventions: A systematic review by John Fouyaxis, Niranjan Bidargaddi, Wei Du, Jeffrey C.L. Looi and Jessica Lipschitz in DIGITAL HEALTH

Supplemental Material

sj-xlsx-2-dhj-10.1177_20552076241306782 - Supplemental material for Critical design decisions and user demographics in enhancing real-time digital mental health interventions: A systematic review

Supplemental material, sj-xlsx-2-dhj-10.1177_20552076241306782 for Critical design decisions and user demographics in enhancing real-time digital mental health interventions: A systematic review by John Fouyaxis, Niranjan Bidargaddi, Wei Du, Jeffrey C.L. Looi and Jessica Lipschitz in DIGITAL HEALTH

Supplemental Material

sj-docx-3-dhj-10.1177_20552076241306782 - Supplemental material for Critical design decisions and user demographics in enhancing real-time digital mental health interventions: A systematic review

Supplemental material, sj-docx-3-dhj-10.1177_20552076241306782 for Critical design decisions and user demographics in enhancing real-time digital mental health interventions: A systematic review by John Fouyaxis, Niranjan Bidargaddi, Wei Du, Jeffrey C.L. Looi and Jessica Lipschitz in DIGITAL HEALTH

Supplemental Material

sj-docx-4-dhj-10.1177_20552076241306782 - Supplemental material for Critical design decisions and user demographics in enhancing real-time digital mental health interventions: A systematic review

Supplemental material, sj-docx-4-dhj-10.1177_20552076241306782 for Critical design decisions and user demographics in enhancing real-time digital mental health interventions: A systematic review by John Fouyaxis, Niranjan Bidargaddi, Wei Du, Jeffrey C.L. Looi and Jessica Lipschitz in DIGITAL HEALTH

Footnotes

Acknowledgements

Dan Thorpe for analysis assistance, Tamara Paget for analysis and editing assistance.

Author contributions

NB conceived the study. JF and NB wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

Consent to participate

Not applicable.

Consent for publication

Not applicable.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical considerations

Not applicable

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

John Fouyaxis

Niranjan Bidargaddi

Supplemental material

Supplemental material for this article is available online.

References

World Health Organization. Mental health [Internet]. [cited 2022 Sep 25]. Available from: https://www.who.int/health-topics/mental-health

World Health Organization. COVID-19 disrupting mental health services in most countries, WHO survey [Internet]. [cited 2022 Sep 25]. Available from: https://www.who.int/news/item/05-10-2020-covid-19-disrupting-mental-health-services-in-most-countries-who-survey

Aboujaoude

Gega

Parish

, et al. Editorial: digital interventions in mental health: current Status and future directions. Front Psychiatry 2020 Feb 27; 11: 111.

Park

. Emerging new era of Mobile health technologies. Healthc Inform Res 2016 Oct; 22: 253–254.

Andersson

. Using the internet to provide cognitive behaviour therapy. Behav Res Ther 2009 Mar 1; 47: 175–180.

Barak

Hen

Boniel-Nissim

, et al. A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J Technol Hum Serv 2008 Jul 3; 26: 109–160.

Hilty

Ferrer

Parish

, et al. The effectiveness of telemental health: a 2013 review. Telemed J E Health 2013 Jun; 19: 444–454.

Verhagen

van Os

Delespaul

. 5 - Ecological momentary assessment and other digital technologies for capturing daily life in mental health. In: Stein

Fineberg

Chamberlain

(eds) Mental health in a digital world [internet]. London, UK: Academic Press, 2022, pp.81–108. (Global Mental Health in Practice).

Lee

. Future of the smartphone for patients and healthcare providers. Healthc Inform Res 2016 Jan; 22: 1–2.

10.

Mosa

ASM

Yoo

Sheets

. A Systematic Review of Healthcare Applications for Smartphones. BMC Med Inform Decis Mak 2012 Jul 10; 12: 67.

11.

Free

Phillips

Watson

, et al. The effectiveness of Mobile-health technologies to improve health care service delivery processes: a systematic review and meta-analysis. PLoS Med 2013 Jan 15; 10: e1001363.

12.

Mohr

Burns

Schueller

, et al. Behavioral intervention technologies: evidence review and recommendations for future research. Gen Hosp Psychiatry 2010 Jul 1; 35: 332–338.

13.

DiFilippo

Huang

Andrade

, et al. The use of mobile apps to improve nutrition outcomes: a systematic literature review. J Telemed Telecare 2015 Jul; 21: 243–253.

14.

Fanning

Mullen

McAuley

. Increasing physical activity with mobile devices: a meta-analysis. J Med Internet Res 2012 Nov 21; 14: e161.

15.

Davies

. ‘Personal health surveillance’: the use of mHealth in healthcare responsibilisation. Public Health Ethics 2021 May 16; 14: 268–280.

16.

Balcombe

De Leo

. Evaluation of the use of digital mental health platforms and interventions: scoping review. Int J Environ Res Public Health 2022 Dec 26; 20: 362.

17.

Firth

Torous

Nicholas

, et al. The efficacy of smartphone-based mental health interventions for depressive symptoms: a meta-analysis of randomized controlled trials. World Psychiatry 2017 Oct; 16: 287–298.

18.

Torous

Bucci

Bell

, et al. The growing field of digital psychiatry: current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry 2021 Oct; 20: 318–335.

19.

Deloitte. Digital Consumer Trends 2020 [Internet]. Deloitte; 2020 [cited 2021 Mar 2] p. 45. Available from: https://images.content.deloitte.com.au/Web/DELOITTEAUSTRALIA/%7B0a975e55-6f49-4aec-b76f-d8317706363b%7D_Digital_Consumer_Trends_2020_Final_0121.pdf?utm_source=eloqua&utm_medium=lp&utm_campaign=20201028-tel-inb-digital-consumer-trends-2020&utm_content=cta&elqTrack=true

20.

Nisha

Iqbal

Rifat

. The changing paradigm of health and Mobile phones: an innovation in the health care system. JGIM 2019 Jan 1; 27: 19–46.

21.

Bidargaddi

Schrader

Klasnja

, et al. Designing m-health interventions for precision mental health support. Transl Psychiatry 2020 Jul 7; 10: 222.

22.

Condon

Coulson

. Chapter 2 - Designing and Delivering Interventions for Health Behavior Change in Adolescents Using Multitechnology Systems: From Identification of Target Behaviors to Implementation. In: Little

Sillence

Joinson

(eds) Behavior Change Research and Theory [Internet]. San Diego: Academic Press, 2017 [cited 2022 Nov 10]. p. 27–45. Available from: https://www.sciencedirect.com/science/article/pii/B9780128026908000013

23.

Heron

Smyth

. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol 2010; 15: 1–39.

24.

Just-in-Time Adaptive Interventions (JITAIs) in Mobile Health : Key Components and Design Principles for Ongoing Health Behavior Support | Annals of Behavioral Medicine | Oxford Academic [Internet]. [cited 2022 Nov 10]. Available from: https://academic.oup.com/abm/article/52/6/446/4733473?login=true

25.

Linardon

Cuijpers

Carlbring

, et al. The efficacy of app-supported smartphone interventions for mental health problems: a meta-analysis of randomized controlled trials. World Psychiatry 2019 Oct; 18: 325–336.

26.

Gilbody

Littlewood

Hewitt

, et al. Computerised cognitive behaviour therapy (cCBT) as treatment for depression in primary care (REEACT trial): large scale pragmatic randomised controlled trial. Br Med J 2015 Nov 11; 351: h5627.

27.

Lipschitz

Hogan

Bauer

, et al. Closing the research-to-practice gap in digital psychiatry: the need to integrate implementation science. J Clin Psychiatry 2019 May 14; 80: 0–0.

28.

Bell

Lim

Rossell

, et al. Ecological momentary assessment and intervention in the treatment of psychotic disorders: a systematic review. PS (Wash DC) 2017 Nov; 68: 1172–1181.

29.

Colombo

Fernández-Álvarez

Patané

, et al. Current state and future directions of technology-based ecological momentary assessment and intervention for Major depressive disorder: a systematic review. J Clin Med 2019 Apr; 8: 465.

30.

Gee B

Griffiths

Gulliver

. Effectiveness of mobile technologies delivering ecological momentary interventions for stress and anxiety: a systematic review. J Am Med Inform Assoc 2016 Jan 1; 23: 221–229.

31.

Chivilgina

Wangmo

Elger

, et al. Mhealth for schizophrenia spectrum disorders management: a systematic review. Int J Soc Psychiatry 2020 Nov 1; 66: 642–665.

32.

Liu

Zhu

, et al. Effects of smartphone-based interventions and monitoring on bipolar disorder: a systematic review and meta-analysis. World J Psychiatry 2020 Nov 19; 10: 272–285.

33.

Balaskas

Schueller

Cox

, et al. Ecological momentary interventions for mental health: a scoping review. PLOS ONE 2021 Mar 11; 16: e0248152.

34.

Riley

Rivera

Atienza

, et al.

Health behavior models in the age of mobile interventions: are our theories up to the task?

Transl Behav Med 2011 Mar; 1: 53–71.

35.

Rosenstock

. The health belief model and preventive health behavior. Health Educ Monogr 1974 Dec 1; 2: 354–386.

36.

Bandura

. Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ, US: Prentice-Hall, Inc, 1986, . xiii, 617 p. (Social foundations of thought and action: A social cognitive theory).

37.

Moher

Liberati

Tetzlaff

, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21; 6: e1000097.

38.

Andrade

. Mean difference, standardized mean difference (SMD), and their use in meta-analysis: as simple as it gets. J Clin Psychiatry 2020 Sep 22; 81: 11349.

39.

Hong

Fàbregues

Bartlett

, et al. The mixed methods appraisal tool (MMAT) version 2018 for information professionals and researchers. Educ Inf 2018 Jan 1; 34: 285–291.

40.

Dulin

Gonzalez

Campbell

. Results of a pilot test of a self-administered smartphone-based treatment system for alcohol use disorders: usability and early outcomes. Subst Abus 2014; 35: 168–175.

41.

Hanssen

Balvert

Oorschot

, et al. An ecological momentary intervention incorporating personalised feedback to improve symptoms and social functioning in schizophrenia spectrum disorders. Psychiatry Res 2020 Feb; 284: 112695.

42.

Prada

Zamberg

Bouillault

, et al. EMOTEO: a smartphone application for monitoring and reducing aversive tension in borderline personality disorder patients, a pilot study. Perspect Psychiatr Care 2017 Oct; 53: 289–298.

43.

Wenze

Armey

Weinstock

, et al. An open trial of a smartphone-assisted, adjunctive intervention to improve treatment adherence in bipolar disorder. J Psychiatr Pract 2016 Nov; 22: 492–504.

44.

Depp

Perivoliotis

Holden

, et al. Single-Session Mobile-augmented intervention in serious mental illness: a three-arm randomized controlled trial. Schizophr Bull 2019 Jun 18; 45: 752–762.

45.

Depp

Ceglowski

Wang

, et al. Augmenting psychoeducation with a mobile intervention for bipolar disorder: a randomized controlled trial. J Affect Disord 2015 Mar 15; 174: 23–30.

46.

Proudfoot

Clarke

Birch

, et al. Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC Psychiatry 2013 Nov 18; 13: 312.

47.

Blevins

Marsh

Stein

, et al. Project CHOICE: choosing healthy options in coping with emotions, an EMA/EMI plus in-person intervention for alcohol use. Subst Abus 2021; 42: 569–576.

48.

Hedman

Andersson

Ljótsson

, et al. Clinical and genetic outcome determinants of internet- and group-based cognitive behavior therapy for social anxiety disorder. Acta Psychiatr Scand 2012 Aug; 126: 126–136.

49.

Rodda

Merkouris

Lavis

, et al. The therapist experience of internet delivered CBT for problem gambling: service integration considerations. Internet Interv 2019 Dec; 18: 100264.

50.

Marzo

Chen

HWJ

Abid

, et al. Adapted digital health literacy and health information seeking behavior among lower income groups in Malaysia during the COVID-19 pandemic. Front Public Health 2022 Sep 14; 10: 998272.

51.

Depression [Internet]. [cited 2022 Nov 10]. Available from: https://www.who.int/news-room/fact-sheets/detail/depression

52.

Mohr

Zhang

Schueller

. Personal sensing: understanding mental health using ubiquitous sensors and machine learning. Annu Rev Clin Psychol 2017 May 8; 13: 23–47.

53.

Kaveladze

Young

Schueller

. Antifragile behavior change through digital health behavior change interventions. JMIR Formative Research 2022 Jun 3; 6: e32571.

54.

Gan

DZQ

McGillivray

Han

, et al. Effect of engagement with digital interventions on mental health outcomes: a systematic review and meta-analysis. Front Digit Health 2021 Nov 4; 3: 764079.

55.

Goldberg

Lam

Simonsson

, et al. Mobile phone-based interventions for mental health: a systematic meta-review of 14 meta-analyses of randomized controlled trials. PLOS Digit Health 2022; 1: e0000002.

56.

Etzelmueller

Vis

Karyotaki

, et al. Effects of internet-based cognitive behavioral therapy in routine care for adults in treatment for depression and anxiety: systematic review and meta-analysis. J Med Internet Res 2020 Aug 31; 22: e18100.

57.

Newby

McKinnon

Kuyken

, et al. Systematic review and meta-analysis of transdiagnostic psychological treatments for anxiety and depressive disorders in adulthood. Clin Psychol Rev 2015 Aug 1; 40: 91–110.

58.

Bear

Edbrooke-Childs

Norton

, et al. Systematic review and meta-analysis: outcomes of routine specialist mental health care for young people with depression and/or anxiety. J Am Acad Child Adolesc Psychiatry 2020 Jul; 59: 810–841.

59.

Cuijpers

. Targets and outcomes of psychotherapies for mental disorders: an overview. World Psychiatry 2019; 18: 276–285.

60.

Moshe

Terhorst

Philippi

, et al. Digital interventions for the treatment of depression: a meta-analytic review. Psychol Bull 2021 Aug; 147: 749–786.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.03 MB

0.07 MB