Abstract
Digital Mental Health Platforms offer feasible options to increase access to mental health support. This study aimed to examine the acceptability of a Low Intensity Cognitive Behaviour Therapy Digital Mental Health Platform, containing a Virtual Coach, with University Students (n = 16) and Mental Health Professionals (n = 5). Semi-structured interviews, exploratory focus groups, and inductive thematic analysis were conducted. Four overarching themes were identified, with potential users and professionals highlighting positive aspects, elements to be improved, and ambivalent feelings towards the platform overall. However, participants predominately expressed negative experiences indicating that the Virtual Coach was unrelatable and hard to engage with. While Virtual Coaches and similar Digital Mental Health Platforms have the potential to overcome barriers for those attempting to access mental health services, their effectiveness may be limited if the people who need them are not drawn to and then consistently engaged with them. Based on the feedback attained for this specific Digital Mental Health Platform, recommendations are provided for future developers aiming to create similar platforms, to assist in their uptake and ensure ongoing user engagement.
Introduction
Given a high number of young people report experiencing anxiety and depression, alongside an inability or unwillingness to access traditional supports, 1 the use of technology for health and wellbeing offers an ideal opportunity as an alternative option to reduce mental illness and increase wellbeing.2,3 Digital Mental Health Platforms (DMHPs) are a relatively new medium for treating mental health conditions and offer the potential to reduce obstacles faced when accessing support, such as stigma, limited access to evidence-informed treatment, and challenges navigating complex and fragmented health systems.4–9 DMHPs are increasingly common, and despite being assigned various names, including ‘relational agent’, 10 ‘automated conversational agent’, 4 and ‘avatar’, 7 they share similar concepts – a virtual agent designed to deliver a mental health program, establish trust, and build a therapeutic alliance with users over time. 10 Due to methodological limitations (e.g. absence of control groups or long-term follow-up data), mixed results on the effectiveness of DMHPs exist.4,8,11 While some studies report that DMHPs can be beneficial, 12 as they may help improve users’ mood scores 11 and reduce depressive symptoms,4,6,7 other studies have reported that individuals may find the platform and their conditions stressful to manage alone, 5 and even lead to increased psychological distress. 13
Participatory research has found growing interest in the ability of avatars and virtual agents to reinforce guidance and support, help set future goals, and fit within a young person’s lifestyle. 14 Despite this interest, the assumption that young, digitally savvy users, or even health professionals, will automatically download, like, and engage with mental health-related apps is misguided. Previous studies consistently report that DMHPs have low levels of engagement and program completion.4,5,11 In terms of general app uptake, it is reported that the primary reason for using an app is social communication and entertainment, with only 5% of general users having a physical or mental health app on their device. 15 In terms of app usage, it is reported that 80% of users who install an app delete it after first use, with only 5% still using it beyond a month. 16 Despite the combination of best technology and evidence-based mental health strategies, this suggests that DMHPs may be redundant if those who need them are not drawn to and then consistently engaged with them.
As a part of another research project, we adapted an existing DMHP – originally developed to support diabetes self-management – to be used by individuals experiencing mild-to-moderate symptoms of anxiety and depression. In conjunction with this, we recognised that there was a valuable opportunity to scope what potential users and professionals thought about it, ultimately gaging their acceptability and potential engagement with the DMHP. The adapted DMHP incorporated a Virtual Coach (VC) with a Low Intensity Cognitive Behaviour Therapy (LICBT) script (referred to as a Low Intensity Virtual Coach (LIVC) from this point forward).
Cognitive Behaviour Therapy (CBT) is a broad umbrella term used to describe a range of techniques that are amongst the most widely employed, scrutinised, and validated forms of psychotherapy. 17 CBT provides a framework for supporting individuals to develop and practice skills needed to recognise and re-assess unhelpful thoughts, dysfunctional appraisals, reduce unwanted emotions, and modify unhelpful behaviours. 18 Low Intensity versus High Intensity (HI) CBT strategies are differentiated by the symptomatology targeted, duration of treatment, session length, and delivery method. LICBT may be used for individuals with mild-to-moderate anxiety or depression and is generally conducted over the phone or internet by LI trained professionals, using a guided self-help booklet over two-to-six sessions. Recent evidence suggests LICBT interventions effectively treat low-to-moderate symptoms of anxiety and depression. 19
The LIVC in the current project sought to reduce users’ mental illness and increase wellbeing, with a virtual coach delivering the LICBT script. In comparison to a standard LICBT approach, a linear and self-determined approach was adopted to suit the platform (e.g. users set their own pace for progression through sessions). This self-determined approach intended to give people control over, and allow them to make choices about, their mental healthcare which is often hindered by systemic and structural factors in traditional health systems. 9 The DMHP included coaching sessions by the avatar, risk checking, homework tasks, monitoring of progress through repeated measurements, and ‘spotlight’ (psychoeducation) sessions on demand.
The current study
Relatively little is known about what makes DMHPs accepted by, and engaging for, people and health professionals. This lack of information represents a considerable barrier to uptake of DMHPs by those who could benefit from them. With this in mind, and having developed a LIVC for individuals experiencing mild-to-moderate anxiety or depression, we sought to explore its acceptability amongst users and mental health professionals to further inform its ongoing development and contribute to the research in this area. Exploratory focus groups with potential users (post graduate students), and semi-structured interviews with mental health professionals, were conducted to investigate what people thought about the look, feel, and functionality of the LIVC, and whether mental health professionals would recommend it to their clients as an adjunct to therapy.
Methods
Participants
A convenience sample of Post Graduate students in the College of Medicine and Public Health (CMPH) at Flinders University, South Australia, and Mental Health Professionals (including psychotherapists and mental health nurses) employed at Southern Adelaide Local Health Network, were invited to participate in the study via email. These email addresses were available to the study authors as a part of the collaborating community. To be eligible to participate in the study, post graduate students had to be current enrolled students in the CMPH, while mental health professionals had to be working in the delivery of mental health/psychological services. Given English was used for the DMHP, interviews and focus groups, individuals who were not fluent in English were excluded from participating. There were no other exclusion criteria. Although the LIVC was designed for individuals experiencing mild-to-moderate symptoms of anxiety or depression, post graduate students were recruited as the aim was to determine what people generally thought about the look, feel, and functionality of the LIVC in this initial pilot phase, not the clinical efficacy of the LIVC. Feasibility constraints limited the ability to include a clinical sample within the required timeframe. Those who were interested in participating in the research, responded to the email invitation, and consequently provided written consent to participate in the research. They were then given details about when and where their focus group or interview would take place.
Procedure
A qualitative exploratory research design was used. One-to-one semi-structured interviews were conducted with participating mental health professionals, while focus groups were conducted with participating post graduate students. Both interviewees and focus group participants were given an iPad to trial the LIVC for 10-to-15 minutes, prior to being asked a series of open-ended questions: ‘What do you think of the app?’; ‘What do you like about it?’; ‘Is there anything you dislike about it?’; ‘Do you think it can be improved?’; and, ‘Would you use this kind of program in a clinical setting?’. Ethics was approved by the Flinders University Social and Behavioural Research Ethics Committee (Research Project Research Project #7871). The researchers followed the Consolidated Criteria for Reporting Qualitative Research (COREQ) guidelines. 20
Analysis
All interviews were digitally recorded, transcribed, and analysed using thematic analysis described by Braun and Clarke. 21 First, the researchers familiarised themselves with the data multiple times. Second, data was manually organised into meaningful findings (N = 141) and assigned initial codes or labels (n = 132 in total), along with relevant extracts. Extracts of data were then coded inclusively to maintain context. Third, the different codes were sorted into potential overarching themes utilising theme-piles, mind-maps, and tables. All relevant coded data extracts were collated within the identified themes. Two researchers (AV and MH) discussed and agreed on the definitions of themes and sub-themes. All material was then reread by the authors to ensure a good fit with extracted themes, and to ensure that themes were internally coherent, consistent, and distinctive. A total of four overarching themes with 16 sub-themes were identified. In line with Braun and Clarke’s 21 method, individual extracts of data were coded and organised into as many different themes as relevant.
Results
In total, four interviews were conducted with five mental health professionals (two mental health nurses, three psychotherapists: age range 33–53 years, 80% female). All interviews were one-on-one, except for one interview which involved two mental health professionals, and on average lasted 25 minutes. One focus group was conducted with 16 Post Graduate students (age range 23–48 years, 94% female) and was 23 minutes long. Figure 1 presents a summary of the thematic analysis, while Table 1 presents a selection of quotations for each theme identified, along with an accompanying qualifier to signify the participant (professional [P] or student [S]). Thematic analysis of the data yielded four overarching themes: (1) positive experiences with the LIVC; (2) negative experiences with the LIVC; (3) ambivalent experiences with the LIVC; and (4) specific suggestions for improvement.
Thematic map of emergent themes and sub-themes derived from thematic content analysis.
Emergent themes and best example response based on analysis of interviews and focus groups.
Theme 1: Positive experiences with the LIVC
The following sub-themes were categorised as positive according to the interviewees’ responses: that it was (a) easy to use; (b) accessible; (c) anonymous, (d) incorporated relevant content; (e) could act as a first step; and (f) was targeted and audience-specific.
Multiple participants commented that they found the LIVC . . .if I had something like that, I would’ve definitely used it, and I was not responsive at all to anyone, for therapy, for anything, I want to do it myself, I don’t want to take advice, I don’t want to be seeing anyone. [S]
Professionals felt that any anxiety around the stigma associated with help-seeking could be reduced by having such an anonymous platform accessed ‘without having to go anywhere’. When prompted to discuss the questions and
Both professionals and students felt that the app could be best utilised as a
Participants’ perceptions of the app’s audience suitability varied considerably, but the consensus was that the app – whilst ‘not for everyone’ [P] – was certainly
Theme 2: Negative experiences with the LIVC
A further, more salient, theme to emerge from the data was an overwhelming sense of participant negativity towards the VC aspect of the LIVC. The following sub-themes were identified by participants as negative aspects of the LIVC generally, and the use of a VC: (a) incongruent; (b) off-putting; (c) annoying; (d) inflexible; (e) unrelatable; and (f) unengaging.
Whilst using the app, professionals had difficulty accepting the VC, stating they ‘wouldn’t be able to take it seriously’ and that ‘it feels wrong’. Professionals struggled to reconcile the use of a digital robot with a wellbeing coach, finding it
Participants not only struggled with the concept of a digital avatar wellbeing coach, but found it, and the app generally, to be ‘
The avatar movements and animations also promoted feelings of . . .You don’t want that situation where you can’t move around the app, and people get stuck with it. . . we’ve tried to jump into a spotlight from anywhere (a spotlight session was psychoeducation focused), and it didn’t let us do it. You’d want to be able to, wherever you are, to be able to get anywhere easily, and quickly, and not get frustrated, and think “Oh, well, I’m stupid because I can’t do it”. [P]
Another professional commented that the app lacked the options and variety necessary to ensure their engagement and continued use. For another professional, the structural rigidity of the automatic answers they had to click on made them feel like they ‘just have to go along with it and be compliant with it’.
Negative experiences with the LIVC also largely transpired as a result of participants finding the VC In terms of therapy, there’s none of that, that therapeutic stuff that comes out of, even phone-based counselling, when talking to someone who’s real on the other end and you get a real emotional response. . . [P]
Just as participants struggled with relatedness and connectedness, they were also predominately
Theme 3: Ambivalent experiences with the LIVC
Aside from the participants who had either positive or negative experiences with the LIVC, were those whose response was ambivalent, for two overarching reasons, that they were either: (a) indifferent; or (b) lost interest.
When prompted by the interviewer, “That’s what you would change? The whole avatar aspect of it”, one professional responded with ‘Yeah, there’s nothing wrong with the questions, they’re fine. . .’. One participant did not feel overly compelled to engage with the program: . . .I’ve used apps in the past for lots of other things, and I’ve gone back into them, “I wonder what’s different” or “I wonder what can I look at”. Whereas, I didn’t feel like I wanted to do that with this one. [P]
When professionals were asked if they would recommend the platform to any of their clients, one stated ‘probably not at this stage’. One student captured this
Theme 4: Suggestions for improvement
When prompted to discuss aspects of the app that interviewees felt could be improved, participant responses centred mainly around two sub-themes: that they wanted to (a) change avatar; and / or (b) increase functionality.
A large amount of criticism was borne from frustrations with the VC component of the program, with several participants making suggestions to
An additional way in which participants discussed the app could be improved was to
Discussion
To investigate the acceptability of a Digital Mental Health Platform (DMHP), that incorporated a scripted Low Intensity Cognitive Behaviour Therapy (LICBT) Virtual Coach (VC) (LIVC), an exploratory focus group and semi-structured interviews were conducted with potential users (post graduate students) and mental health professionals. Inductive thematic analysis was used to identify four overarching themes and 16 sub-themes. Results indicated that despite strong evidence for LICBT, 19 and the growing interest in and promise of online tools for mental health support, 22 both potential users and mental health professionals predominately indicated that they did not relate to the LIVC. In particular, participants generally expressed dislike for the feel and functionality of the LIVC and expressed concerns around a digital avatar being unable to achieve what a professional can in face-to-face therapy. While positive comments were noted on the accessibility, anonymity, and appropriateness of the LIVC for youth, both potential users and professionals predominately indicated that they found the LIVC hard to engage with.
The current findings are consistent with other studies, particularly one examining a computerised Cognitive Behaviour Therapy (cCBT) program for depression in a UK primary care setting. 5 Knowles et al. 5 found that participant’s experiences when using cCBT were on a continuum, and could be broadly classified as ‘positive’, ‘negative’ or ‘ambivalent’. Whilst participants valued the online cCBT program for enhanced privacy, flexibility and autonomy, others struggled with the absence of interpersonal contact. Consistent with this, participants in the current study struggled with the lack of ‘rapport’, ‘expressions’, ‘emotions’ and ability to ‘form a therapeutic alliance’ with the LIVC.
The inability of DMHPs to incorporate interpersonal factors such as visual body language, empathy, warmth, and congruence experienced through the therapeutic alliance may be problematic for their ongoing use and effectiveness. These common factors have been reported to correlate more strongly with client outcomes than actual intervention treatments, 23 a finding which has also been reported in studies of adolescents, 24 underscoring the potential limitations of DMHP’s in their absence. Consistent with this, previous research suggests that young people have a strong need for personal connection in the context of mental healthcare. 9 Future research should explore how a balance may be achieved between the valued aspects of DMHPs, such as privacy, autonomy and control, 25 and important interpersonal factors which are achieved through the therapeutic alliance in traditional treatment settings.
The two main suggestions put forth by participants were to improve the DMHP’s functionality and to change the avatar. Participants commented that the DMHP could have been structured differently (e.g. easier to use, better paced) to make it more engaging from first impressions. These findings are similar to a clinical review of user engagement with mental health smartphone apps, which suggested poor usability can be a significant factor leading to poor engagement, uptake, and prolonged utilisation, 22 and highlights the challenge of integrating structured evidence-based therapies with engaging and flexible ‘apps’. Moreover, it may be that the intention to balance a humanesque/cartoon-like avatar to enhance user engagement inadvertently led to the opposite. Participants commented that the body movements and tone of voice made them ‘uncomfortable’ and that they thought it was ‘hard to relate to’ and ‘strange’, with participants suggesting that either videos of real people, or just text, would be more relatable. Participants’ dislike for the avatar may be explained in relation to the concept of the ‘uncanny valley’. 26 The ‘uncanny valley’ proposes that objects which almost perfectly resemble human beings, with some shortcomings, provoke uncanny or strangely familiar feelings or discomfort in observers. 27 With learnings from the ‘uncanny valley’ in mind, unless an appropriate balance of humanness is achieved to increase engagement with DMHP’s, then the ideal of providing scalable digital mental health support to all who need it, when they need it, may remain elusive.
Limitations
Data were limited to the experiences of a small, predominantly female, convenience sample drawn from a University and health setting in South Australia. A different sampling strategy, greater number of participants, and a clinical treatment seeking, rather than non-symptomatic population, may have led to different observations as the LIVC was designed for people with mild-to-moderate depression and anxiety. Further research is necessary to determine whether the themes identified generalise to users from all backgrounds, ages, and locations, and whether similar feedback would be gathered on similar DMHPs with a LIVC. Moreover, as participants had ⩽15 minutes to experience the look and feel of the LIVC, they may have not been exposed to its full functionality, such as the ability to remember personal data, receive push notifications, and select between avatars. The adapted LICBT approach incorporated into the LIVC programs offered may have been too ‘clinical’ for the platform or audience. However, it is reported that only a small portion of the over 10,000 mental health ‘apps’ available for download actually contain evidence-based behaviour change techniques, as compared to pure ‘feel good’ activities. 28 Therefore, the evidence-based approach incorporated into the LIVC is seen as a strength of the current study, rather than a limitation. While valuable information about enhancing users’ acceptability of, and engagement with, a VC has been provided, in line with research by Orlowski et al. 9 the results indicate that top-down development processes are not optimal for ensuring user acceptability.
Conclusion
An existing Digital Mental Health Platform (DMHP) was adapted to create a novel Low Intensity Virtual Coach (LIVC) to alleviate some of the common obstacles to accessing help for mental health conditions. In contrast to expectations, potential users and mental health professionals’ feelings towards and engagement with the LIVC were mixed. Results suggest that despite the best intentions, available technology, and evidence-based content, DMHPs may be ineffective if the people who need them are not drawn to and consistently engaged with them.
Footnotes
Declaration of conflicting interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors were involved in the design of the LIVC.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethics
Ethics was approved by the Flinders University Social & Behavioural Research Ethics Committee (Research Project #7871).
