Sage Journals: Discover world-class research

Abstract

Background

With the aim of improving treatment retention in patients with the onset of alcohol-related liver disease (ArLD), we designed a blended intervention (brief motivational intervention + ‘serious game’ (SG)). We present the participatory design methodology and outcomes and the usability assessment of the intervention.

Methods

(1) The design of the SG was based on the outcomes of two 3-h co-creation sessions with 37 participants (healthcare and technology professionals, patients, and patients’ relatives). The brief face-to-face motivational intervention was based on the 5 As Model and adapted to the ArLD population. (2) Usability pilot study: 20 participants (10 ArLD patients + 10 healthcare professionals) received the intervention. System Usability Scale (SUS) and Post-Study System Usability Questionnaire (PSSUQ) were applied to assess the SG usability and patients’ satisfaction with it. Weekly semi-structured interviews on the phone were conducted to identify the preferred elements in the SG and those aspects that should be improved.

Results

(1) Design: an SG in the form of a gamified web app, consisting of a daily activity for six weeks and adapted brief motivational interviewing. (2) Usability pilot study: usability results were excellent for both patients and healthcare professionals (SUS median score = 85). The general usability, the quality of the information provided by the SG and the quality of the interface were very positively rated in the PSSUQ (overall median score = 2, IQR = 1–2). The best-rated aspects were the provision of feedback, the use of metaphors and the application of audiovisual material. Changes in the design, response mechanics and content were applied after the study.

Conclusions

The usability and acceptability of an intervention for increasing retention to treatment in patients with recent onset of ArLD and AUD were excellent for patients and healthcare professionals. A randomized-controlled trial is required to test the efficacy of this approach.

Keywords

Digital health adherence retention to treatment alcohol use disorder alcohol liver disease serious games MI

Introduction

Alcohol use is one of the leading preventable risk factors contributing to the global burden of disease.^1,2 Chronic alcohol use is the main cause of liver disease,^3–5 and in the European Union, 41% of liver deaths are attributed to alcohol.⁶ Heavy alcohol use over a sustained period of years has been described to lead to the highest risk of alcohol-related liver disease (ArLD), and these patients receive a disproportionate percentage of liver disease diagnoses,³ even light-moderate alcohol consumption (12–24 g per day) leads to ArLD. ArLD includes alcoholic fatty liver, alcoholic steatohepatitis, cirrhosis, hepatocellular carcinoma, or alcoholic hepatitis.⁴

Patients with ArLD require an integrated multidisciplinary care model,^7,8 for the management of alcohol use as well as nutritional and pharmacological aspects.⁹ Addressing alcohol use is crucial to control the progression of liver disease. Achieving alcohol abstinence is fundamental as it induces clinical improvement in any of the ArLD stages.¹⁰ Abstinence has been described as the strongest determinant for the improvement of long-term outcomes in these patients, while alcohol relapse has been described as the strongest predictor of readmission to hospital and mortality.^7,11–14

Retention to addiction treatments predicts outcomes of ArLD and alcohol use disorder (AUD).^14–17 In ArLD early alcohol rehabilitation in an inpatient or outpatient addiction treatment after hospital discharge, reduces relapses, readmissions and mortality.¹⁴ Adherence to pharmacological treatment has been related to lower withdrawal symptoms and lower relapse rates in patients with AUD.¹⁶

Despite the impact of treatment retention on treatment compliance and treatment efficacy, adherence to the addiction treatment is scarce in patients consulting for an organic pathology, especially in patients with ArLD.^9,13 For instance, in a cohort of 120 patients who survived alcoholic hepatitis requiring hospital admission, only 58% started treatment in the addiction unit and 37% were still on treatment after one year.¹⁸ Many reasons explain the poor adherence. Patients’ perceived stigma towards the disease is one of the most described ones.¹⁹ Public stigma, self-stigma, and structural stigma have been described to increase the overall burden of ArLD by contributing to failure or delays in seeking help, inferior healthcare and negative health outcomes,^19,20 and shifting blame to the individuals.²¹ Other reasons for decreased adherence include the denial of the disease,²² poor insight into AUD and the need for treatment (little acceptance that AUD can be treated, or absence of physical symptoms), misconceptions about the treatment of alcoholism or lack of social support systems.^7,22–25 Therefore, patients with an ArLD represent a susceptible population to receive specific interventions to improve treatment adherence to achieve abstinence. However, there are few studies in these patients on pharmacological or psychological interventions to improve adherence and prognosis of AUD. Motivational interviewing (MI) and cognitive behavioural therapy (CBT) have shown promise in this regard,²⁶ and studies suggest that integrated (‘one-site’) treatment between hepatology and addiction units is essential.^25,27–29

MI is a short-term, directive, patient-centred communication style focused on resolving ambivalence and concluding in a habit change.³⁰ It is characterised by its flexibility and a comprehensive approach to the individual (not focused on health problems but on the person). The four processes on which the spirit of MI is based are partnership between the therapist and the patient, evoking patients’ individual reasons for behavioural change, emphasising patient autonomy (acceptance), and compassion. The main skills in MI would be open questions, affirming, reflecting, exchanging information, and summarising.³⁰ MI has shown efficacy in reducing the use of alcohol and other substances, enhancing adherence to pharmacological treatment (i.e. HIV, diabetes mellitus, asthma, etc.)²⁶ and in improving retention in patients with AUD.^26,31,32

CBT is one of the gold-standard treatments for AUD.^25,33 CBT is a time-limited, multisession intervention targeting cognitive, affective, and environmental factors related to substance use.³³ It involves techniques to identify and modify dysfunctional beliefs, maladaptive patterns of thinking and behavioural habits, and providing coping skills to prevent continued alcohol use, and achieve abstinence or harm reduction.^33–35 In addition, serious games (SGs) have also shown promise in improving treatment adherence. SG consist of the application of game design (e.g. videogames) in a context and with a purpose other than pure entertainment looking for the achievement of a positive behavioural change outside the game, and the necessary motivation and improvement in the learning of skills. In health, SG has shown efficacy in improving depressive symptoms, posttraumatic stress, autism, attention-deficit/hyperactivity disorder, and cognitive impairment. Moreover, interventions using SG to promote the treatment of mental health disorders have shown feasibility and high levels of acceptability among users and providers.^36,37 In relation to alcohol, SG has been used to increase knowledge regarding the consequences of alcohol consumption in adolescents, craving³⁸ and in reducing attentional bias.³⁹ So far, few studies have explored the efficacy of SG to reduce alcohol intake, maintain abstinence, or adherence to addiction treatment.^40,41 To our knowledge, only one study has explored the efficacy of an SG to practice relapse prevention strategies.^41,42 Of note, SG has shown promise in improving treatment adherence in other pathologies.⁴³ No studies have yet explored the combination of a face-to-face intervention based on MI and CBT with an SG in patients with an ArLD.

The design of effective SGs needs to consider several key elements.⁴⁴ First, engagement, which can be enhanced by incorporating strategies such as immediate goals like level completion, the use of points as a measure of success or memorable and adapted content as an access reward. Second, the accessibility of SGs and their use without supervision need to be considered. To increase accessibility, multi-platform SG has been defined as the gold standard. Third, SG needs to be consistent with the treatment the patient is receiving and based on well-established psychological approaches. SG should translate to real-world situations to achieve generalisation of the skills practised. To achieve effective skill building, activities in the game should be patient-centred, to its performance or situation.⁴⁴ Last, for SGs to be effective, it is recommended to involve final users from the design phase.⁴⁴

Taking together the essential role of abstinence in patients with an ArLD, the low adherence and retention rates of these patients to addiction treatment and the potential of blended intervention (SG + MI), the aim of this project is to develop an intervention consisting of a brief motivational session combined with an SG to improve retention to the outpatient addiction treatment. The SG would offer a gamified and engaging platform, aligned with MI and CBT principles, fostering a supportive environment for users. The aim is to provide psychoeducational information on addiction treatment, as well as skills to achieve behavioural change (even before the patient has had the first contact with the addiction unit) to clarify misinformation that could hinder their attendance to the visits.

Here, we describe the intervention design process (Phase 1: co-creation) and what does the final intervention consist of. We report the results from the usability study (Phase 2: usability pilot study).

Methods

In the present section we describe the methodology applied for the design and development of the intervention (brief MI and SG) and for the usability pilot study.

Development and functions of the intervention

Brief motivational intervention

The brief face-to-face intervention to be deployed when patients are included in the study was designed by the research team, whose members are experts in MI and have experience in treating patients with ArLD. The objective was to design a brief intervention aiming to make a first approach to the AUD and invite the patient to join the Addictions Unit treatment as well as to use the digital intervention (SG) created for the project. It is an adaptation from the 5 As Model,⁴⁵ which is based on MI, and fits with the standard approach in healthcare. In addition, the last step in the 5 As Model allows to offer arrangements to follow up, which coincides with the aim of the project.^45,46

The model was adapted following the characteristics and needs of patients with an ArLD according to the co-creation process as well (see ‘My Way Up’: development, co-creation Session 1), and incorporating the MI spirit (partnership, evocation, acceptance, compassion) and skills (open-questions, affirmations, reflections, summaries, attending to the language of change and exchange of information).³⁰

‘My Way Up’: Development

The design of the SG intervention was conducted using a co-creation and participatory approach,⁴⁷ and based on well-established principles from MI, CBT, and game-based learning.

Two online 3-h co-creation sessions based on the Shape Up method^© (Cookiebox) were conducted with the participation of 37 professionals from different fields (mental health, addiction, hepatology, internal medicine, digital health) and professions (medicine, psychology, nurses, social work, SG designers); as well as patients that had received the diagnose of ArLD and their relatives. The group was formed by five patients, two caregivers, three doctors from internal medicine, five psychologists, seven psychiatrists, six hepatologists, two nurses, three social workers, two researchers, and two experts in digital health. Participants were grouped into five heterogeneous groups. Participants had access to a shared document in which they were presented with short guided activities (some individual and some to be completed in the organised small groups) that allowed them to frame the gap in treating ArLD and treatment components that could be useful. These activities allowed us to collect information in the shared document that defines the present situation, the desired situation, and the elements that the digital tool should include.

Session 1 Analysis Phase of the SG (understanding): analyse the characteristics of the standard treatment (what works, what should be improved, barriers that professionals face, what should be achieved using the SG); and the characteristics of the patients that will receive the intervention (which is their usual technology usage, what motivates them, which behaviours should be acquired, which reduced, and which maintained).

Session 2 Synthesis Phase of the SG (co-creating the solution): defines the elements that should be incorporated in the SG and which format should it consist of. In this session, participants described the characteristics they would like to include (i.e. competition vs. cooperation, realistic vs. fictional; etc.); which game elements would be interesting (challenges, narrative, extra content, medals, social status, etc.); memorable activities that should guide the modules (acquiring knowledge, mastering abilities, having fun, competing, etc.). Responses from all the groups were merged, and those elements that held more consensus were the ones considered for the development of the intervention. At the end of the session, each group presented an idea for the metaphor that should lead the SG, and one was chosen by consensus among the participants.

The elements that might apply should be linked to our patient's engagement besides to our objectives: signposting mechanics to correctly guide them; progress and feedback indicators that guarantee tracking while avoiding potential moments of aversion; an intelligent expert system that automates consequences to decisions within the memorable experience; level and progression measurements that are visualised; learning and reporting certificates, notifications and badges; access to item collections and complete sets of access, status and power rewarding systems where the engagement is intrinsically driven.

Information from both sessions was put together in a report document. Gaps and strengths in the standard treatment were defined from the responses in the activities from Session 1. When the activities were completed, a group consensus was achieved regarding the most relevant topics the intervention should incorporate (brainstorming and then voting). All participant responses to the activities were aggregated in a document, the most repeated topics were given priority in the design by the research team of the content to be included in the modules. The gamification characteristics that were highlighted the most in Session 2 were the ones applied in the game. With the information obtained in both sessions, the research team worked together with the developers and game designers to create the structure and modules of the SG and to offer guidance to ensure that principles from MI and CBT were being properly applied.

Pilot study: Usability and acceptability of the intervention

Participants and procedure

To study the usability of the intervention (SG + brief intervention), we conducted a pilot study among a sample of 20 participants: 10 patients who had received the diagnosis of ArLD and had been referred to the Addiction Unit at the Hospital Clínic of Barcelona; 10 healthcare professionals from different fields (addictions, mental health, hepatology, internal medicine) and professions (psychiatrist, psychologist, nurses, social workers, medicine) that work with the target patients of the intervention. The patients who participated in the design phase, as well as five additional patients (recruited in their follow-up visit with the principal investigator of the study), were asked to participate in the usability study. The 10 professionals that participated were a convenience sample selected from the participants in the design phase maintaining a representation of all the professional profiles (professional specialty and working unit). They were recruited at the Hospital Clinic de Barcelona and Hospital Germans Trias i Pujol.

The participants received the face-to-face intervention, and at the last step (offer arrange to follow-up), they received an individual anonymised code and password to access the SG intervention and were asked to complete the six modules (one module per week) plus onboarding.

Participants were asked to complete the usability questionnaires by accessing a link at the end of the SG. In addition, one weekly semi-structured interview was conducted over the phone (total of six interviews/participant), regarding the experience on the SG during the previous week. With these interviews, we aimed at identifying the elements that were more relevant for the participants as well as those that would need to be improved. We conducted these interviews weekly as, besides exploring aesthetic or technical elements of the SG, the coherence, clarity, and adequacy of the content in each module (one module/week) was explored, and thus some of the questions were memory-dependent.

Usability measures

The global usability of the intervention and the participants’ satisfaction with it were measured at the end of the SG intervention (as indicated in the questionnaires) with adapted versions of:

- The System Usability Scale (SUS)⁴⁸: 10 items rated in a five-point Likert scale (from 1 – strongly disagree, to 5 – strongly agree). A higher SUS score indicates better usability. SUS scores are interpreted as follows: higher than 80.3 as excellent, higher than 68 to 80.3 as good, 68 as okay, 51 to 68 as poor and below 51 as awful.

- The third version of the Post Study System Usability Questionnaire (PSSUQ)⁴⁹: 16 items rated on a seven-point Likert scale (1 – strongly agree, to 7 – strongly disagree). The PSSUQ offers an overall usability score, and three additional scores: system usefulness, information quality and interface quality. A lower score in the four PSSUQ scores indicates higher perceived usability.⁵⁰ Three additional items were added to the PSSUQ to assess whether the participants considered that it would be interesting to be able to share the information from the SG with healthcare professionals; whether they considered that the SG would help improve treatment retention; and if they had enjoyed the design of the game interface (i.e. colours, font, drawings, etc.).

The weekly semi-structured interview assessed the following aspects regarding the usability of the content and weekly modules (see Supplemental Material 1, S1):

- Content of the module: aim of the module (whether participants were able to remember it), adequacy of the content covered, and adjustment of the activities proposed to the content.

- Most valued elements of the module: which activity did they find more relevant, which (at least) three aspects they enjoyed the most, and which resource they preferred (i.e. type of response, visual content, extra content, etc.).

- Elements to be improved: technical problems or difficulties, web appearance, any content or activity that was repetitive or slowed the rhythm of the experience, and reduced engagement to continue, response format, content they missed or would be appropriate for the module.

- Elements related to the form of use of the SG: platform in which they used it, frequency of usage, pattern of use (i.e. completing all the activities the same day, waiting until the end of the week to complete the module, etc.) and reason for the pattern (i.e. forget to use), and average time to complete the activities in the module.

Statistical analysis

The primary outcome in our study is the usability of the SG intervention defined as the global score in the SUS. The analysis of the SUS score was conducted according to the scoring strategy of Brooke.⁴⁸ The results for the usability have been analysed using the median and inter-quartile ranges. Results from the two additional questions added to the PSSUQ are presented in percentages (patients’ responses will be classified as ‘highly agree or agree’, ‘neutral’ or ‘disagree or highly disagree’). To analyse differences in the usability scores between patients and professionals, we have used Mann–Whitney U.

The results from the semi-structured phone interviews have been analysed qualitatively. The themes regarding the SG were specified by the structure of the review: content; most enjoyed aspects; less enjoyed ones; and pattern of use of the SG. Starting from these three big topics, and according to the participant answers, we defined sub-topics (i.e. technical aspects; aesthetics; response dynamics). For each subtopic, as answers to the interview were quite structured (i.e. specify the three aspects you liked the most about this week module), it was possible to summarise all participant responses and provide a list of topics for each question. We considered the most valued elements in the SG those that were repeated more often by the participants. The elements to be improved are presented together with the solution applied.

Ethical considerations

The present project has been approved by the Clinical Research Ethics Committee of the Hospital Clinic of Barcelona (HCB/2020/0378). All the data were collected and stored in encrypted and secure servers belonging to Hospital Clinic de Barcelona, following the guidelines and standards of the 2018 European General Data Protection Regulation, and were only managed by the project researchers.

Results

The analyses of the comments provided by the participants in the first co-creation session, allowed to describe the present situation in the therapeutic approach and the ideal situation (including the expected outcomes from the project) (see Table 1). Building from these comments, the objectives that the intervention should cover were defined.

Table 1.

Outcomes from the analysis phase: understanding the current situation and describing the future desired scenario.

Present treatment approach

Elements that work: trust relationship free of judgement, making the patients feel accompanied in the therapeutic process; empathy; joint definition of the therapeutic objectives; multidisciplinary approach (coordination between hepatology and addictology); motivational approach; fast and efficient derivation from the addiction unit to hepatology; paying attention not only to problems related to alcohol use; clarifying the role of alcohol on the liver disease and prognosis
To be improved: attendance to the first visit in the addiction unit; paternalism; too many days between visits which hinders adherence; stigma and shame in the patients regarding their problem with alcohol; access to treatment should be improved; maintenance of abstinence and prevention of relapses; low knowledge of resources in the environment of patients that could improve their well-being
What frustrates the professionals: the fact that the patient does not benefit from treatments that could improve their well-being and quality of life; readmissions due to worsening of the liver condition; continued drinking despite the severity of the liver disease; intervention that is not efficient
Outcomes of the present approach: heterogeneous, some patients adhere easily to the addiction treatment while others are lost at follow-up

Desired situation

Goals to achieve with the new intervention: improvement in treatment retention (retention for the longest period of time); attendance to control visits both in the addiction and liver units; reduced alcohol relapse; improved abstinence rates (and acknowledgement that their well-being improves when quitting alcohol); improve prognosis and survival of the patients; stimulate a changed behaviour providing information on their health problem as well as strategies to achieve healthier habits; early attendance to the addiction unit; improvement of patients’ self-efficacy and autonomy (provide small challenges to be solved, positive reinforcement, evoke options and alternatives for the patient to choose an action plan, allow to see the evolution); provide strategies to manage anxiety; the serious game should be seen as a link with the hospital and the treatment; offer the possibility to reflect on their alcohol use with a reduction in stigma; optimise the resources provided by the hospital

Brief motivational intervention

The resulting brief motivational intervention consists of a session, structured following the five steps from the five As Model, as follows:

Assessment: the pattern of alcohol use is explored, as well as eating and exercise habits, quality of life (EQ-5D-5L),⁵¹ and functionality (Functioning Assessment Short Test).⁵²

Ask for permission to give advice: offer personalised feedback on the need of changing habits, reflecting on the impact on quality of life and functionality of the existing habits.

Explain that they will receive guidelines from liver nurses regarding eating and exercise habits adapted to their liver condition.

Offer feedback regarding the patients’ alcohol pattern of use emphasising that there is no safe use after the onset of liver alterations.

Explore the patients’ opinions on the need to change habits and abandon alcohol use.

Explore the patients’ opinions on the need of follow-up in the addictions unit and how these professionals can help improve their liver condition, quality of life, and well-being.

Agree on individual goals: define the stage of change in which the patient is at.

Listen actively and empathically, without judging to define the willingness of patients to incorporate the required habit changes. Open questions and reflections can be useful.

Assist patients regarding their goals: elicit the patient, or propose with the patient's permission, a menu of alternatives to achieve the goals of changing habits (including reducing alcohol use).

Open questions to evoke ways in which the patient could change his usual behaviour pattern.

Offer some alternatives: communicate (inform the closest people on their decision to cut down alcohol use or to prepare a strategy to reject alcoholic drinks when offered); substitute alcoholic drinks for other type of drinks, etc.

Remind patients of the role that the addiction unit plays in their recovery.

Explore the patients’ opinions on the proposed alternatives and plan how they will carry out the decided alternatives.

Offer arrange to follow-up: offer a first visit to the addiction unit as well as the possibility to use the SG (if they are allocated to the experimental group).

‘My Way Up’: Results from the co-creative design

With the information extracted from the two sessions, the research team outlined the priority contents to be included in the SG. The result has been the ‘My Way Up’ web app. It is accessible through any technological device, and it is gamified, meaning that participants need to complete an adventure inside the experience, and it contains elements such as personalised feedback, points and medals, and interactive and visual elements (see Figure 1). Because storytelling is crucial in both Motivational Design and SGs, there is a metaphor guiding the game in which an online avatar (which represents the patient) aims to climb a mountain up to the ‘Wellness Peak’. Patients are told that they will have to overcome a different scenario each week (i.e., crossing a river, walking through quicksand, etc.) by completing a simple daily activity (seven activities/week). Also, each week they receive the support of a ‘mountain expert’ that corresponds to a healthcare professional (hepatologist, psychiatrist, psychologist, nurse, social worker, and social educator). The content of the SG has been divided into six modules, that are arranged following the four processes in MI (Table 2).

Figure 1.

Screenshots from the ‘My Way Up’ serious game. At the top, from left to right: layout of the six weeks; activity consisting of a crossword; points and medal received after completing the seven activities of the week. At the bottom, from left to right: positive reinforcement message after successfully completing the activity. At the bottom of the screen, the compass drawing signalling the easter egg is visible; the activity consisting of ranking how true each statement is for the patient; log-in screen and logo.

Table 2.

‘My Way Up’ modules.

MI Basic process	Phase in the game (weeks)	Activities included
Engaging	Reasons that may have led to the debut of the disease	Beliefs about the disease; beliefs about the factors that led to the diagnosis Impact of the disease on daily functioning Changes already applied after the diagnosis Things the patient enjoys doing Important values for the patient Important people supporting the patient What do they expect from the treatment and the SG
Engaging + focusing	Aspects affecting the disease, and which can they influence	Identify day-to-day difficulties, and which of these are most important Which difficulties may be related to the liver problem and which to alcohol consumption How does the liver disease affect their day-to-day life What do they think they can do to feel better, resources they have to feel better, and what things have already been implemented Introduction of healthy lifestyle habits
Focusing + evoking	Tools to improve well-being through the enhancement of healthy lifestyles	Identify which professionals and people around the patient can offer support Available abilities and tools in the patient to achieve better well-being: obstacles to applying these abilities Habit change and the importance of habits in daily life Advantages of alcohol use cessation Aspects that the patient will need to control in his/her life
Evoking	Change of habits and benefits of change	Identify behaviours (and subsequent emotional reactions) Relationship between behavioural change and habit change Strategies they have used to achieve habit change and advantages and disadvantages of these strategies
Evoking	Resources to achieve change	Levels of perception in habit change: self-perception, change of strategies and change of results Areas where they could begin to implement changes (diet, exercise, alcohol use, etc.) Definition of changes to be incorporated in the defined areas and the strengths available to carry them out Management of internal (anger, frustration, etc.) and external (conflicts with other people, work stress, etc.) conflicts and how they affect behaviour. People to turn to. Progress (not focused on reaching the objective but on the continuity of the intervention)
Planning	Planification of the change	Define which professionals will participate in their treatment Define the reference persons to whom to turn to in case of difficulty Stimulus control strategies Identify the internal and external situations that may hinder change, and propose solutions Final review of values important to the patient, their objectives, ways to achieve them, obstacles, and solutions

MI: motivational interviewing; SG: serious game.

The modules and activities in the SG have a strong psychoeducational approach aiming to improve health literacy regarding ALD and addiction treatment concepts. It incorporates MI spirit and skills, and concepts from CBT such as the associations between internal and external stimuli, thoughts, emotions and actions; strategies to promote behavioural changes; and stimulus control strategies. This content, which arose from the outcomes that were ranked as most relevant in co-creation Session 1, is presented using gamified activities within the metaphor of the game, expecting a higher motivation in the patient to complete the modules and ultimately generate a higher retention in the addiction treatment.

The web app and its activities are designed as a sequence of ‘great-to-deal-with’ challenges in conjunction with constant measurement. This allows to offer interesting and surprisingly new challenges. In fact, the design incorporates two key elements: a storytelling layer that allows participants to get involved besides a full and real customisation control of the activities.

At the beginning of the SG, there is an ‘onboarding’ activity, that provides information regarding the objective of the SG (with a video that presents the story within the game) to be completed through one daily activity during six weeks. Participants are also presented with the mechanics that will be used within the activities. Mechanics are motivational triggers in the form of game elements behind the tasks to be performed that connect behavioural objectives with the patient's interests. At the end of the SG, patients can choose from three options how their ‘Wellness Peak’ looks and receive a written message from the research team and a final video acknowledging the work conducted during the six weeks and inviting to continue the process.

With access to the SG, the patient receives a notebook that can act as a journal where they can write down the answers to some of the activities in the SG, in case they find it easier to consult afterwards.

‘My Way Up’ has been developed using the ‘Tacket by CookieBox’ tool,⁵³ that offers interactive narratives in which the user can visualise and interact with the content. This tool has a visible part for the users, and another only visible for the developer that can be configured (adding text, images, graphic design, interactive activities, etc.). The access to the ‘SG’ is through a random code of numbers and letters, with no relation to any personal data of the patient that could allow the identification. ‘My Way Up’ and the data resulting from its use, are stored in a remote server from the Hospital Clínic de Barcelona. The SG does not collect data from the patients’ devices, and only the information strictly related to the SG activities will be used. No personal or clinical data from the patient will be included or collected in the SG. The servers needed for the data storage are Apache or Nginx, PHP 7 and Mysql. The minimum server capacity is 2CPU with 2GB RAM and 20GB HD.

Pilot study: Usability and acceptability of the intervention

The sample consisted of 20 participants, 10 patients and 10 healthcare professionals. Eleven women and nine men. The participants in the professional group were from the following fields and professions: one hepatologist, two internists, two nurses specialised in addiction, one nurse with a background in hepatology, two psychiatrists, one psychologist, and one mHealth specialist.

The usability of the ‘My Way Up’ web app was rated with a median score of 85 (IQR = 71.88–90) in the SUS, which can be interpreted as ‘excellent’.⁴⁸ Eighty-five percent of the participants rated the usability of the intervention as ‘excellent’ (n = 10) or ‘good’ (n = 7). No significant differences were found for the median usability score between healthcare professionals and the group of patients (median for patients = 85, IQR = 70–90; median for professionals = 85, IQR = 77–90; T = 0.16; p = .87). Median scores for the 20 participants for each of the 10 questions from the SUS can be found in Table 3.

Table 3.

System Usability Scale.

Item	Total median (IQR)	Patients median	Professionals median
I would use the web app frequently	3 (3–4)	4 (4–5)	3 (3–3)
I found the web app unnecessarily complex	1 (1–2)	2 (1–4)	1 (1–2)
I thought the web app was easy to use	5 (4–5)	5 (4–5)	5 (5–5)
I think that I would need the support of a technical person to be able to use this web app	1 (1–3)	1 (1–3)	1 (1–3)
I found the various functions in this web app were well-integrated	4 (4–5)	4 (4–4)	5 (5–5)
I thought there was too much inconsistency in this web app.	1.5 (1–3)	1 (1–2)	2 (1–3)
I would imagine that most people would learn to use this web app very quickly	4 (3–5)	4 (4–5)	4 (3–5)
I found the web app very cumbersome to use	1 (1–1)	1 (1–2)	1 (1–1)
I felt very confident using the web app	5 (4–5)	5 (4–5)	5 (5–5)
I needed to learn a lot of things before I could get going with this system	1 (1–1)	1 (1–1)	1 (1–1)

IQR = interquartile range

*Rating score range from 1 = totally disagree to 5 = totally agree.

PSSUQ indicated a high perceived usability of the SG among both healthcare professionals and patients. The overall score, as well as the other three subscales indicated that the usability of the intervention, the quality of the information provided and the SGs’ interface quality were positively perceived by the participants (Table 4). No statistically significant differences were found between professionals and patients for the scoring in any of the PSSUQ scales.

Table 4.

Post-system usability questionnaire.

Scale^a	Global median (IQR)	Patients’ median (IQR)	Professionals’ median (IQR)	Statistic (p-value)^b
Overall	2 (1–2)	2 (2–2)	2 (1–2)	45 (.73)
System usability	1.5 (1–2)	2 (1–2.5)	1 (1–2.5)	42 (.93)
Information quality	2 (1–2.5)	2 (1–2.5)	2 (1–2)	40 (1)
Interface quality	2 (1–2.75)	2 (1–3)	2 (1–2)	50 (.44)

IQR = interquartile range; ArLD = alcohol-related liver disease.

Rating score from 1 = best to 7 = worst.

Median differences between scores from the healthcare professionals and patients with ArLD.

Regarding the three questions added to the PSSUQ, most participants considered the SG as a positive tool to improve treatment retention in patients with ArLD (75% of the participants totally or strongly agree with this, n = 15; and 20% were quite in agreement, n = 4). 90% of the participants considered that a positive added value to the SG would be being able to share the progress in the web app with healthcare professionals (12 participants totally agreed with this, 60%; six strongly agreed, 30%; and two were quite in agreement, 10%). Regarding the question of whether they had enjoyed the design of the interface (colours, drawings, distribution of the elements in the screen), 75% of the participants totally (n = 7) or strongly (n = 8) agreed, while 15% (n = 3) quite agreed.

The weekly semi-structured interviews explored three main themes (S1):

Content: participants could recall the objective of each week, both the aim within the narrative (i.e. crossing a river), and the objective of the module. It was highly valued that the content of the SG did not focus solely on alcohol use, but also incorporated information regarding other habits and psychological well-being, and participants indicated that this allowed them to reflect on relevant and valued aspects of their lives. For example, a hepatologist indicated that ‘this week I have realised that the activities allowed me to reflect on aspects that were causing me anxiety’, or a patient when describing an activity on the associations between emotion and thoughts explained ‘it helps asking yourself questions’; another patient explained ‘I enjoyed learning that depending on your attitude towards a situation you may act differently … it helped me reflect on the consequences of my actions’. Also, providing with strategies and resources to achieve behavioural changes was rated positively ‘it is very useful when you have to quit drinking to get some ideas of things that can help face the new situation’, ‘I could learn on relaxation techniques, and ways of plan how to achieve a change in my diet’.

The inclusion of extra content was also highlighted. This content would normally be presented in the form of an ‘easter egg’ or secret content and presents with some information that complements the content in the module. For example a patient on a video from the movie ‘Inside out’ ‘I really enjoyed the video regarding how emotions work … it was a fun way to understand it’.

Most valued aspects of the intervention: the most valued aspects of the SG would focus either on aesthetics; game mechanics; and the narrative.

Regarding aesthetics, the inclusion of visual elements (memes, videos, pictures) was highly valued, especially the content in the easter eggs (as indicated earlier).

The use of metaphors within the narrative in the game was the most repeated item. The narrative following the metaphor of ascending a mountain was followed throughout the experience. Also, activities within the game would incorporate metaphors to facilitate understanding of other concepts. A psychiatrist indicated that ‘I highly enjoyed the activity representing who want to be a millionaire to reflect on the people who the patient may have around to support him’.

Regarding game mechanics, participants valued several aspects:

- The SG offered different answers depending on their responses (personalised feedback).

- A variety of response mechanics (rating their agreement with certain affirmations, multiple choice, emojis, blank space to fill, changing the order of some sentences…) was considered to ease the motivation towards the activities. ‘it is good that the format of the activities changes, it makes it more fun’.

On the other hand, the weekly semi-structured interviews allowed us to identify several components and mechanics of the SG that needed to be improved (Table 5). Participants indicated that it did not take more than 5 min to complete each daily activity.

Table 5.

Identified issues in the SG and solutions.

Identified issue	Solutions
Bugs and programming errors	Were fixed by the team of programmers.
Design: Lack of uniformity in the design style. Difficulties to read due to size of the font or lack of contrast with the background. Difficult to identify the compass drawing signalling the presence of an easter egg. Difficulty to estimate the duration of the daily activities (i.e. screens completed from the total of the activity).	The design was unified throughout the activities and weeks: titles in white, instructions and information in the activities in black, and bold for the most relevant aspects. The size of the font was increased in many activities. Colour of the font has been improved to ensure that it can be read easily and contrasts sufficiently with the background colour. The compass drawing was lightly circled in red to facilitate its detection by participants. A progress bar was added at the top of the screen within the daily activities, so patients could estimate their progress.
Response mechanics Difficult to know the selected option within the activities. Drag and drop activities would be confusing as options would change position in a non-intuitive way.	Improvements to make them more intuitive: Response options will increase in size and change colour when the participant moves the cursor over it. When an option is selected, it will change its colour. Activities consisting of sorting several options (drag and drop): the already sorted options will change in colour. Whenever one option is rated, the others would just descend one position.
Content Instructions not clear enough. Some content could fall short for patients who are motivated to learn more. Repetitive content. The activities in which one could reflect on their own experience were the most valued ones.	Extra content was added regarding change of habits (week 4), as healthcare professionals pointed out that some patients could be motivated to learn more about health recommendations for their condition. Change of wording in some activities to improve comprehension. Some content on habit change was found repetitive so it was summarised in less activities and other content was added. Activities with an open space to reflect on their own experience were included.
Audiovisuals Lack of relevance of some digital contents or adequacy to participants’ references. Some pictures or memes were distracting, or their meaning was difficult to understand.	The mindfulness training video was changed as some participants found it too difficult to follow for beginners. Some memes were updated for others considered more appropriate.

Discussion

The current study details the development and usability assessment of an intervention combining a brief motivational intervention with the use of a digital SG for the improvement of retention to addiction treatment in patients presenting with an ArLD. This approach is a pioneer in the treatment of ArLD.

The usability of the SG intervention was rated very positively by the participants in the study with no differences found between healthcare professionals or patients with an ArLD. Results suggest that the SG is easy and simple to use, the dynamics of the SG were simple to understand, no inconsistencies were found, and users did not need support to use it. Furthermore, participants were able to easily complete the activities within the web app, found it easy to understand the mechanics, and they easily solved difficulties that appeared within the activities. Also, the information provided in the combined intervention was simple to understand, well-organised, and clearly allowed them to complete the activities in the SG. The interface was pleasant, and they enjoyed using the web app. The item that received the lower score in the SUS referred to the willingness to use the SG frequently. This result seems to be explained by the fact that professionals tended to be neutral in their responses as they are not the intervention target, while patients scored more positively.

It is noteworthy that the SG was considered by most participants as a promising tool to improve treatment retention in patients diagnosed with an ArLD. For future versions of the web app, we would also consider the interest of both healthcare professionals and patients in being able to share with them the results from the activities in the game.

A strength in our project is the fact that the design of the ‘SG’ has been conducted using a co-creative participatory approach. This implies that end users have an active role in all the stages of the design process: ideation and gathering insights, prototype development, implementation and evaluation.^47,54,55 Usually, end users are involved in the ideation phase through the participation in focus groups, in which they provide valuable feedback to frame the problem and possible solutions to it, but are not involved in further steps of the design process. The involvement of the patients has been described as useful to improve the satisfaction of patients attending mental health services. This approach is coherent with MI, as both consider the patient as an expert about their own illness and care, allow for a better and increased understanding of mental distress and incorporate the patients’ perspective in the comprehension of the disease.^37,47 Involving healthcare professionals that treat the patient from different perspectives and at different moments of the disease allows to incorporate in the intervention of those elements that are considered crucial when receiving the diagnosis, as well as identifying on the one hand what usually works in healthcare practice and in the other those aspects that should be improved. Furthermore, these are the professionals that will be recommending the use of the SG, and therefore is important to include them as well in the design and evaluation of the tool. Taking all these together, we consider that applying this methodology should lead to an intervention that is adapted to the needs of future users and therefore improves adherence.

The final product of the design phase has been a blended intervention that combines a brief face-to-face intervention designed by professionals in addiction with proven skills in MI, together with a six-week SG web-based intervention. We believe that the intervention has been improved after the study to make it more usable and relevant. Both components include a motivational approach, focusing on how patients can contribute to their own process of change (coinciding with MI and CBT approaches). The face-to-face intervention and the SG, with their psychoeducational approach, aim to overcome misconceptions regarding the disease and the treatment as well as to enhance insight regarding AUD, which are barriers to treatment retention. The motivational approach allows patients to reflect on the factors that affect their own decision-making and take a more active role in their process of behaviour change. On the other hand, the incorporation of CBT elements pretends helping the patients identify the chains of behaviour needed to achieve their health goals, as well as to recognise the interactions between thoughts, emotions and actions, and provides with structured strategies to move towards the acquisition of new habits. This information is provided using a gamification strategy, including challenging tasks, decision-making scenarios, and adaptive learning features, contributing to skill development and empowerment, as well as motivation towards its use. Moreover, users gain problem-solving skills, resilience, and self-efficacy, which can extend beyond addiction treatment. Moreover, it is noteworthy that the designed intervention is cost-effective, and could be of use in healthcare centres that lack a specialised addiction unit.

The six-week duration of the SG has its strengths and weaknesses. It may be somewhat ambitious and could pose challenges for compliance among less motivated patients or those with less technological literacy. However, on the positive side, it enables patients to initiate thinking about their therapeutic goals and plan for lifestyle changes. Additionally, using the SG during the time gap of two weeks to a month between receiving the brief face-to-face intervention and the first visit with the addiction psychiatrist, allows for patients to take an active role, giving them the opportunity to learn more about their health situation.

The next step in the project will be to assess the efficacy of the intervention to improve retention in the outpatient addiction treatment. For that, we will conduct a randomised controlled trial (RCT) among 68 patients. Half will receive the experimental intervention (motivational brief intervention + SG) and the other half will receive treatment as usual. All of them will be offered a monthly visit for six months. In addition to treatment retention, other secondary outcomes will be assessed, including addiction treatment adherence, quality of life and functionality, reduction in alcohol use, days of abstinence, liver disease evolution and motivation towards treatment (see the complete protocol for the RCT in supplementary materials, S2). Engagement with the SG will be also studied to explore its role in the study outcomes.

Several limitations of our study need to be outlined. First, the sample included is small, which also limits the interpretation of the obtained statistical values. However, we expect that this limitation is solved by the design methodology and by the qualitative analyses. The sample was multidisciplinary as it included patients and professionals from different fields, with broad knowledge of the disease with the aim of precisely identifying the patients’ profile and needs. Second, in the present pilot study, we did not assess information regarding retention/adherence, liver disease progression, and alcohol use pattern, and neither data regarding adherence to the intervention. We will collect all these variables in the next phase of the project (RCT). Digital literacy could limit the use of our tool in certain patients, however, we considered this aspect when designing the SG and incorporated certain hints (i.e. tutorial at the beginning of the experience; and dynamics that facilitate response). Last, we received negative feedback in 1–2 patients. Nevertheless, these negative scores contrasted significantly with the scores of the rest of the participants. We also hope that changes after incorporating feedback, including those who provided negative feedback, have improved the tool.

Conclusions

The usability and acceptability of a blended intervention (brief MI plus ‘My Way Up’ SG) for improving treatment retention in patients with recent onset of ArLD and AUD has been described as excellent among patients and healthcare professionals. An RCT trial is required to test the efficacy of this specific approach.

Supplemental Material

sj-docx-1-dhj-10.1177_20552076241242787 - Supplemental material for Multiplatform web app (My Way Up) plus motivational interviewing for improving treatment retention in patients with onset of alcohol-related liver disease and alcohol use disorder – an example of participatory research

Supplemental material, sj-docx-1-dhj-10.1177_20552076241242787 for Multiplatform web app (My Way Up) plus motivational interviewing for improving treatment retention in patients with onset of alcohol-related liver disease and alcohol use disorder – an example of participatory research by Elsa Caballeria, Mercè Balcells-Oliveró, Ramón Bataller, Pol Bruguera, Noel Cabrera, Alexandra Estruch, Neus Freixa, Óscar Garcia-Pañella, Mireia Graell, Jordi Gratacós-Gines, Pablo Guzman, Anna Hernández-Rubio, Anna Lligoña, Martina Pérez-Guasch, María Teresa Pons-Cabrera, Elisa Pose, Paola Zuluaga and Hugo López-Pelayo in DIGITAL HEALTH

Supplemental Material

sj-docx-2-dhj-10.1177_20552076241242787 - Supplemental material for Multiplatform web app (My Way Up) plus motivational interviewing for improving treatment retention in patients with onset of alcohol-related liver disease and alcohol use disorder – an example of participatory research

Supplemental material, sj-docx-2-dhj-10.1177_20552076241242787 for Multiplatform web app (My Way Up) plus motivational interviewing for improving treatment retention in patients with onset of alcohol-related liver disease and alcohol use disorder – an example of participatory research by Elsa Caballeria, Mercè Balcells-Oliveró, Ramón Bataller, Pol Bruguera, Noel Cabrera, Alexandra Estruch, Neus Freixa, Óscar Garcia-Pañella, Mireia Graell, Jordi Gratacós-Gines, Pablo Guzman, Anna Hernández-Rubio, Anna Lligoña, Martina Pérez-Guasch, María Teresa Pons-Cabrera, Elisa Pose, Paola Zuluaga and Hugo López-Pelayo in DIGITAL HEALTH

Footnotes

Acknowledgments

We are grateful to all participants. The project is supported by a grant from the Instituto de Salud Carlos III (ISCIII) and co-funded by the European Union.

Conflicts of interest

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Ramón Bataller: consultancy with GlaxoSmithKline, Novo Nordisk and Boehringer ingelheim; speakers’ bureau: AbbVie and Gilead. The rest of authors have no conflicts to declare in relation to the work presented in this article.

Contributorship

Study planning, project conception and coordination were conducted by HLP and EC; all authors contributed to the design of the intervention; recruitment and data extraction were conducted by EC and HLP. EC and HLP were involved in the manuscript preparation. All authors revised the manuscript.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This project (PI20/00760) was funded by the Instituto de Salud Carlos III (ISCIII) and co-funded by the European Union.

Guarantor

HL-P.

Supplemental material

Supplemental material for this article is available online.

ORCID iDs

Elsa Caballeria

Anna Hernández-Rubio

Jordi Gratacós-Gines

References

Murray

CJL

Aravkin

Zheng

, et al. Global burden of 87 risk factors in 204 countries and territories, 1990–2019: a systematic analysis for the global burden of disease study 2019. Lancet 2020; 396: 1223–1249.

WHO. Global status report on alcohol and health 2018. WHO, http://www.who.int/substance_abuse/publications/global_alcohol_report/gsr_2018/en/ (2019, accessed 15 October 2020).

Rehm

Guiraud

Poulnais

, et al. Alcohol dependence and very high risk level of alcohol consumption: a life-threatening and debilitating disease. Addict Biol 2018; 23: 961–968.

Seitz

Bataller

Cortez-Pinto

, et al. Alcoholic liver disease. Nat Rev Dis Prim 2018; 4: 1–22.

Rehm

Shield

. Global burden of alcohol use disorders and alcohol liver disease. Biomedicines 7. Epub ahead of print 1 December 2019. DOI: https://doi.org/10.3390/BIOMEDICINES7040099.

Thursz

Gual

Lackner

, et al. EASL clinical practice guidelines: management of alcohol-related liver disease. J Hepatol 2018; 69: 154–181.

DiMartini

Leggio

Singal

. Barriers to the management of alcohol use disorder and alcohol-associated liver disease: strategies to implement integrated care models. Lancet Gastroenterol Hepatol 2022; 7: 186–195.

Casanova

Bataller

. Alcoholic hepatitis: prognosis and treatment. Gastroenterol Hepatol 2014; 37: 262–268.

Bataller

Cabezas

Aller

, et al. Alcohol-related liver disease. Clinical practice guidelines. Consensus document sponsored by AEEH. Gastroenterol Hepatol 2019; 42: 657–676.

10.

Addolorato

Burra

Burt

, et al. Guías de Práctica Clínica de EASL: Tratamiento de la Hepatopatía Alcohólica. 2012.

11.

Singal

Bataller

Ahn

, et al. ACG clinical guideline: alcoholic liver disease. Am J Gastroenterol 2018; 113: 175.

12.

Kodali

Kaif

Tariq

, et al. Alcohol relapse after liver transplantation for alcoholic cirrhosis—impact on liver graft and patient survival: a meta-analysis. Alcohol Alcohol 2018; 53: 166–172.

13.

Altamirano

López-Pelayo

Michelena

, et al. Alcohol abstinence in patients surviving an episode of alcoholic hepatitis: prediction and impact on long-term survival. Hepatology. Epub ahead of print June 2017. DOI: https://doi.org/10.1002/hep.29338.

14.

Peeraphatdit

(Bee), Kamath

Karpyak

, et al. Alcohol rehabilitation within 30 days of hospital discharge is associated with reduced readmission, relapse, and death in patients with alcoholic hepatitis. Clin Gastroenterol Hepatol 2020; 18: 477–485.e5.

15.

Aguiar

Neto

Lambaz

, et al. Prognostic factors during outpatient treatment for alcohol dependence: cohort study with 6 months of treatment follow-up. Alcohol Alcohol 2012; 47: 702–710.

16.

Zweben

Pettinati

Weiss

, et al. Relationship between medication adherence and treatment outcomes: the COMBINE study. Alcohol Clin Exp Res 2008; 32: 1661–1669.

17.

Lohit

Kulkarni

Galgali

. Factors influencing adherence to anti-craving medications and drinking outcomes in patients with alcohol dependence: a hospital-based study. J Pharmacol Pharmacother 2016; 7: 72.

18.

López-Pelayo

Miquel

Altamirano

, et al. Treatment retention in a specialized alcohol programme after an episode of alcoholic hepatitis: impact on alcohol relapse. J Psychosom Res 2019; 116: 75–82.

19.

Schomerus

Leonhard

Manthey

, et al. The stigma of alcohol-related liver disease and its impact on healthcare. J Hepatol 2022; 77: 516–524.

20.

Kilian

Manthey

Carr

, et al. Stigmatization of people with alcohol use disorders: an updated systematic review of population studies. Alcohol Clin Exp Res 2021; 45: 899–911.

21.

The Lancet Gastroenterology & Hepatology. From stigma to solutions: fighting for effective alcohol policy. Lancet Gastroenterol Hepatol 2024; 9: 1.

22.

Mellinger

Scott Winder

DeJonckheere

, et al. Misconceptions, preferences and barriers to alcohol use disorder treatment in alcohol-related cirrhosis. J Subst Abuse Treat 2018; 91: 20–27.

23.

Horrell

Callaghan

Dhanda

. Alcohol misuse in patients with alcohol-related liver disease: how can we do better? A narrative review of the literature. Alcohol Clin Exp Res 2022; 46: 1364–1370.

24.

Saunders

Zygowicz

Angelo

, et al. Person-related and treatment-related barriers to alcohol treatment. J Subst Abuse Treat 2006; 30: 261–270.

25.

Leggio

Lee

. Treatment of alcohol use disorder in patients with alcoholic liver disease. Am J Med 2017; 130: 124–134.

26.

Palacio

Garay

Langer

, et al. Motivational interviewing improves medication adherence: a systematic review and meta-analysis. J Gen Intern Med 2016; 31: 929–940.

27.

Caputo

Domenicali

Bernardi

. Diagnosis and treatment of alcohol use disorder in patients with end-stage alcoholic liver disease. Hepatology 2019; 70: 410–417.

28.

Glance

Dimartini

. Identification and management of alcohol dependence and withdrawal in alcoholic liver disease. Clin Liver Dis 2013; 2: 57.

29.

Lucey

Singal

. Integrated treatment of alcohol use disorder in patients with alcohol-associated liver disease: an evolving story. Hepatology 2020; 71: 1891–1893.

30.

Miller

Rollnick

. Motivational interviewing: preparing people for change. 2nd ed, https://psycnet.apa.org/record/2002-02948-000 (2002, accessed 10 August 2021).

31.

Bachiller

Grau-López

Barral

, et al. Motivational interviewing group at inpatient detoxification, its influence in maintaining abstinence and treatment retention after discharge. Adicciones 2015; 27: 109–118.

32.

Lawrence

Fulbrook

Somerset

, et al. Motivational interviewing to enhance treatment attendance in mental health settings: a systematic review and meta-analysis. J Psychiatr Ment Health Nurs 2017; 24: 699–718.

33.

Magill

Ray

Kiluk

, et al. A meta-analysis of cognitive-behavioral therapy for alcohol or other drug use disorders: treatment efficacy by contrast condition. J Consult Clin Psychol 2019; 87: 1093.

34.

Liese

Beck

. Back to basics: fundamental cognitive therapy skills for keeping drug-dependent individuals in treatment. NIDA Res Monogr 1997; 165: 207–232.

35.

Judith

. Terapia Cognitivo-Comportamental - Teoria e Prática. 2022; 1999: 1–6.

36.

Mccallum

. Gamification and serious games for personalized health. Stud Health Technol Inform. Epub ahead of print 2012. DOI: https://doi.org/10.3233/978-1-61499-069-7-85.

37.

Fitzgerald

Ratcliffe

. Serious games, gamification, and serious mental illness: a scoping review. Psychiatr Serv 2020; 71: 170–183.

38.

Martínez-Miranda

Espinosa-Curiel

. Serious games supporting the prevention and treatment of alcohol and drug consumption in youth: scoping review. JMIR Serious Games 2022; 10: e39086.

39.

Flaudias

Zerhouni

Chakroun-Baggioni

, et al. Reducing attentional bias in individuals with alcohol use disorders with a tablet application: A randomized controlled trial pilot study. Alcohol Alcohol. Epub ahead of print November 2019. DOI: https://doi.org/10.1093/alcalc/agz080.

40.

Fleming

Bavin

Stasiak

, et al. Serious games and gamification for mental health: current status and promising directions. Front Psychiatry 2017; 7: 215.

41.

Lau

Smit

Fleming

, et al. Serious games for mental health: are they accessible, feasible, and effective? A systematic review and meta-analysis. Front Psychiatry 2016; 7: 1.

42.

Verduin

LaRowe

Myrick

, et al. Computer simulation games as an adjunct for treatment in male veterans with alcohol use disorder. J Subst Abuse Treat 2013; 44: 316–322.

43.

Höchsmann

Infanger

Klenk

, et al. Effectiveness of a behavior change technique–based smartphone game to improve intrinsic motivation and physical activity adherence in patients with type 2 diabetes: randomized controlled trial. J Med Internet Res 21. Epub ahead of print February 2019. DOI: https://doi.org/10.2196/11444.

44.

Shah

Kraemer

Won

, et al. Developing digital intervention games for mental disorders: a review. Games Health J 2018; 7: 213–224.

45.

World Health Organization. Toolkit for delivering the 5A’s and 5R’s brief tobacco interventions in primary care, https://apps.who.int/iris/bitstream/handle/10665/112835/9789241506953_eng.pdf (2014, accessed 20 August 2021).

46.

Friedman

Lyna

Sendak

, et al. Use of the 5 As for teen alcohol use. Clin Pediatr (Phila) 2016; 56: 419–426.

47.

Tait

Lester

. Encouraging user involvement in mental health services. Adv Psychiatr Treat 2005; 11: 168–175.

48.

Brooke john. SUS: A ‘Quick and Dirty’ Usability Scale. 1996; 207–212.

49.

Lewis

. Psychometric evaluation of the PSSUQ using data from five years of usability studies. Int J Hum Comput Interact 2002; 14: 463–488.. Http://DxDoiOrg/101080/1044731820029669130 .

50.

Sauro

Lewis

JR.

Quantifying the User Experience: Practical Statistics for User Research .

51.

Badia

Roset

Montserrat

, et al. La versión española del EuroQol: descripción y aplicaciones. Med Clin (Barc) 1999; 112: 79–86.

52.

Rosa

Sánchez-Moreno

Martínez-Aran

, et al. Validity and reliability of the functioning assessment short test (FAST) in bipolar disorder. Clin Pract Epidemiol Ment Health 2007; 3: 5.

53.

Cookie Box - Leading European Gamification, https://cookiebox.es/ (accessed 29 January 2023).

54.

Martínez-Cañas

Ruiz-Palomino

Linuesa-Langreo

, et al. Consumer participation in co-creation: an enlightening model of causes and effects based on ethical values and transcendent motives. Front Psychol 7. Epub ahead of print 26 May 2016. DOI: https://doi.org/10.3389/fpsyg.2016.00793.

55.

Simpson

House

. Involving users in the delivery and evaluation of mental health services: systematic review. Br Med J 2002; 325: 1265–1268.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.01 MB

0.05 MB