Approaches for implementing digital interventions for alcohol use disorders in primary care: A qualitative,user-centered design study

Abstract

Background

Digital interventions, such as smartphone apps, can be effective in treating alcohol use disorders (AUD). However, efforts to integrate digital interventions into primary care have been challenging. To inform successful implementation, we sought to understand how patients and clinicians preferred to use apps in routine primary care.

Methods

This study combined user-centered design and qualitative research methods, interviewing 18 primary care patients with AUD and nine primary care clinicians on topics such as prior experiences with digital tools, and design preferences regarding approaches for offering apps for AUD in primary care. Interviews were recorded and transcribed for template analysis whereby a priori codes were based on interview topics and refined through iterative coding. New codes and cross-cutting themes emerged from the data.

Results

Patient participants with AUD indicated they would be more likely to engage in treatment if primary care team members were involved in their use of apps. They also preferred to see clinicians “invested” and recommended that clinicians ask about app use and progress during follow-up appointments or check-ins. Clinician participants valued the opportunity to offer apps to their patients but noted that workflows would need to be tailored to individual patient needs. Time pressures, implementation complexity, and lack of appropriate staffing were cited as barriers. Clinicians proposed concrete solutions (e.g., education, tools, and staffing models) that could improve their ability to use apps within the constraints of primary care and suggested that some patients could potentially use apps without clinician support.

Conclusions

A user-centered approach to engaging patients in digital alcohol interventions in primary care may require personalized support for both initiation and follow-up. Meeting patients’ needs likely require increased staffing and efficient workflows in primary care. Health systems should consider offering multiple pathways for enrolling patients in apps to accommodate individual preferences and contextual barriers.

Plain Language Summary

Healthcare systems have begun using app-based treatments to help patients manage their health conditions, including alcohol use disorders. Some apps have been tested in research studies and appear to be effective. However, it is difficult for healthcare teams to offer apps to patients. Clinicians must engage in new activities that they have not done before, such as “teaching” patients to use apps and checking in on their use of the apps. Identifying how to use apps in routine healthcare is critical to their successful implementation. This study interviewed 27 people, including healthcare providers and patients in primary care, to uncover the most optimal ways to offer apps to patients with alcohol use disorders. The interviews combined the use of qualitative research methods and user-centered design. Results suggest that to use to address alcohol use disorders, primary care teams should be prepared to offer personalized support to patients. Both patient and clinician interviewees said that the steps required to use apps must be intuitive and simple. Patients could gain more benefits if clinicians introduced the apps and guided patients to use them, as opposed to making apps available for patients to download and use on their own. However, the exact approach to offering apps would depend on a given patient’s preferences and the extent that staffing was available in the clinic to support patients. Health systems should be prepared to offer and support patients in their use of apps, which should accommodate patient preferences and the constraints of the clinic.

Keywords

alcohol use disorders mHealth digital interventions primary care implementation science

Introduction

Alcohol use disorders (AUDs) are prevalent, costly, and life-threatening (Center for Disease Control and Prevention, 2015a, 2015b; Degenhardt & Hall, 2012; Glass et al., 2010; Grant et al., 2015; Mokdad et al., 2004; National Institute on Alcohol Abuse and Alcoholism, 2007). Only eight in 100 people with past-year AUD receive treatment. Most people with AUD prefer to receive help for their drinking in primary care, rather than in specialty treatment programs (Barry et al., 2016). Experts have called for increased treatment of AUD in primary care (Crowley et al., 2015; Ghitza & Tai, 2014; Institute of Medicine, 2006; National Council for Mental Wellbeing, 2018; Saitz et al., 2008). Given the time constraints of primary care and historical treatment of AUD in specialty programs often outside healthcare settings, health systems may need to adopt new approaches to deliver alcohol treatment to a large number of people.

Digital interventions, such as smartphone apps that deliver behavioral interventions, have been a proposed solution to increase access to behavioral health treatment in primary care (Quanbeck et al., 2014; Ramsey et al., 2019). They often incorporate a range of behavioral counseling strategies, such as advice, skills training, and feedback (Hermes et al., 2019; Kiluk et al., 2019; Quanbeck et al., 2018; Ramsey et al., 2019). Some apps have features that facilitate their integration into healthcare, such as clinician portals that include “population management” features so clinicians can access and monitor patient progress and self-reported outcomes. Meta-analytic evidence suggests that digital interventions can reduce alcohol consumption (Kaner et al., 2017; Kiluk et al., 2019), and several apps designed for AUD treatment are supported by evidence for their efficacy and effectiveness (Bickel et al., 2008; Campbell et al., 2014; Christensen et al., 2014; Glass et al., 2017; Gustafson et al., 2014; Kiluk et al., 2016; Marsch et al., 2014).

Yet, there is still a lack of knowledge about how to integrate digital interventions into routine healthcare (Auerbach, 2019; Mares et al., 2016; Thies et al., 2017). Researchers and health system leaders face important decisions when designing services that enable delivery and use of digital interventions at the point of care (Hermes et al., 2019; Mohr et al., 2017). For instance, patients need support to effectively engage in digital interventions. This may require healthcare teams to conduct new activities that are not routine, such as “teaching” patients to use software, longitudinally monitoring patients’ software use, and conducting follow-up to help them feel supported and accountable to using the software (Glass et al., 2021; Hermes et al., 2019; Mohr et al., 2011).

Given digital interventions are still new to healthcare, more knowledge is needed to inform workflow design, including understanding critical perspectives of patients with AUD and primary care clinicians to help guide the implementation of digital interventions and the processes in clinical care that would enable their delivery. This study sought to solicit perspectives of patients and clinicians to uncover implementation barriers and facilitators, and to inform the design of workflows that could be used when implementing treatment apps for AUD in primary care. Engaging patient and clinician stakeholders to develop a user-centered approach for supporting the use of apps for AUD in primary care has the potential to increase implementation success. Both qualitative research and user-centered design methods (International Organization for Standardization, 2019) are well suited to understanding stakeholder needs. Therefore, this study combined these methods to elicit and describe patient and clinician preferences for integrating apps for AUD in primary care.

Methods

Setting

This study was conducted at Kaiser Permanente Washington (KPWA), an integrated health system with approximately 700,000 enrollees. All primary care clinics have implemented population-based screening for alcohol use. Care in this system included brief alcohol interventions, referral to integrated mental health specialists, and alcohol medications (Glass et al., 2018). Specialist addiction treatment was available via referral.

Study Phases and Recruitment

This study had two phases, conducted from November 2018 to May 2019. The first phase involved semi-structured interviews with patients recruited from nine KPWA primary care clinics in the Seattle, WA, USA metropolitan area. Patient participants were eligible if they were ≥18 years old, used a smartphone at least weekly, and met criteria for AUD per a structured diagnostic interview based on the Diagnostic and Statistical Manual of Mental Disorders (fifth edition) (Sheehan et al., 1997, 1998). We identified potentially eligible patient participants by accessing information about alcohol use recorded in electronic health records. Supplemental Appendix A details procedures for participant recruitment, identification, and screening processes. To establish the sample size for patient participants, we began analyzing data while the interviews were underway and monitored for data saturation (Saunders et al., 2018).

The second phase involved semi-structured interviews with information-rich clinician participants who helped facilitate integration of mental healthcare in KPWA, including primary care-based social workers, primary care providers (PCP), nurses, medical assistants, and community resource specialists We sought to interview as many clinicians who agreed to participate; 19 were invited.

The KPWA Institutional Review Board approved all study procedures. Study staff obtained written informed consent from all participants who enrolled. The IRB issued a waiver of consent to identify potential participants, and a waiver of documentation of consent to assess eligibility via telephone.

Semi-Structured Interviews

Interview guides were similar for patients and clinicians. Key topics covered prior experiences with digital tools in healthcare; experiences, attitudes, expectations, and needs for care related to AUD, and design preferences regarding approaches for offering apps for AUD in primary care. Questions about the approach for offering apps were informed by balance theory (e.g., individuals, tasks, and tools that could facilitate the experience; Carayon, 2009), implementation science frameworks for digital interventions (e.g., the extent of clinician support, how to guide patients to use apps) (Hermes et al., 2019), and prior digital intervention studies (e.g., privacy issues, barriers to accessing patient self-reported data) (Carayon, 2009; Knowles et al., 2015; Lapham et al., 2012; Mares et al., 2016; O’Connor et al., 2016). Supplemental Appendix B contains the interview guides.

In conjunction with the interview guides, the interviews employed methods of user-centered design (International Organization for Standardization, 2019) to elicit preferences and actionable feedback on features of candidate approaches for offering and supporting the delivery of apps for AUD. This included asking participants to react to low-fidelity prototypes that communicated ideas and concepts about hypothetical workflow designs in the form of storyboards. The storyboards (Truong et al., 2006) illustrated core features of alternative approaches to integrating alcohol treatment apps into primary care, and reflected three main phases of patient engagement: (a) introducing apps for AUD as a treatment option, (b) supporting patients to successfully setup the app and initiate its use, and (c) supporting patients with follow-up to help them to stay engaged in using the app. For each of these phases, the storyboards presented three to four optional approaches as a basis for eliciting feedback. The stories varied in how much help the patient received, which primary care team members were involved, and whether communication was by phone, in-person, or secure messaging through an online health portal. We grounded storyboards in personas depicting different “user profiles” for hypothetical patients with AUD symptoms who might be interested in learning about options for help. After showing the storyboards to participants and asking for their perceptions, we asked them to rank order the options by preference and probed for additional input. Supplemental Appendix C contains the personas and storyboards.

To provide participants with first-hand knowledge about apps, participants were given a functional alcohol treatment app and a “getting started guide” with instructions for initiating the app. Clinicians were shown a two-min video created by the study team that provided an overview of the app. Interviewers also administered a demographics questionnaire. Patients completed the self-report Quick Drinking Screen (Sobell et al., 2003).

Two researchers with graduate-level research training in qualitative and user-centered design methods conducted the interviews. To promote interview consistency, materials were first piloted with volunteers and each interviewer shadowed one another along with the principal investigator (PI). Throughout the pilot interviews and during the research interviews, the PI listened to audio recordings and read transcripts, providing feedback as necessary. The interview team met regularly to discuss results and to iteratively adapt the interview materials.

Interviews were digitally recorded and professionally transcribed, then imported into Dedoose v7.0.23 for coding (SocioCultural Research Consultants, 2018). We used the method of template analysis (King, 2004), blending inductive and deductive qualitative analysis methods. A priori codes were based on storyboard phases and interview topics. Transcripts were analyzed to identify key phrases or units of meaning to allow new codes to emerge from the data and to adjust the a priori codes. For instance, our initial codebook had a first-level code “workflow” and three second-level codes, “intro,” “setup,” “follow-up,” to capture content related to the storyboard phases described above. Early analyses suggested that perspectives on the workflow were better captured by changing these second-level codes to represent workflow activities (e.g., “learning about the treatment app,” “installing and setting up the app,” “app walk-through,” “follow-up with the primary care team,” and “getting technical help”) instead of using the a priori linear phases. After the patient and clinician interview templates were finalized, we assessed inter-coder agreement (MacQueen et al., 1998) and discussed discrepancies until acceptable reliability was achieved (Cohen's κ ≥ 80%) (Bernard & Ryan, 2002). Interviews for six patients and three primary care team members were double-coded and analyzed to establish κ ≥ 80% for each set of participants. To enhance rigor (Creswell, 2013; Padgett, 2008), we kept memos and an audit trail to document decisions, and met frequently to review memos and coding. Two senior investigators (sociology and general internal medicine) reviewed the template and representative excerpts along with decisions to modify or supplement the interview guide (Graneheim & Lundman, 2004). After the data were coded, we examined for themes occurring within codes, and then across the coding hierarchy (“cross-cutting” themes) (King, 2004). After initial themes were identified, the lead interviewer and PI used affinity mapping, a user-centered design approach to cluster or group similar ideas or concepts together, to discuss and reach consensus about themes (Beyer & Holtzblatt, 1998).

Patient and clinician interviews were analyzed separately. Workflow preferences were summarized within patient and clinician samples and then compared. Themes and representative quotes were presented in research meetings before being finalized.

Results

Participants

Eighteen patients (P1–P18) and nine clinicians (C1–C9) completed interviews (Table 1). Twelve patients had previously received some type of care for AUD. Fourteen patients had previously used health apps. Of the clinicians we interviewed, five were social workers who were embedded in primary care teams, two were PCPs, one was a medical assistant, and one was a community resource specialist.

Table 1.

Participant characteristics.

Patient characteristics (n = 18)	N or mean (range)
Age in years
Less than 29 years	6
30–39 years	6
40–49 years	2
50–59 years	2
60 or older	2
Gender
Men	13
Women	5
Black, indigenous, or person of color (BIPOC)	7
College degree	9
Employed	14
Standard drinks per week past 30 days	22 (0–77)
Heavy episodic drinking days past 30 days	10 (0–30)
AUDIT-C score	8 (3–11)
Alcohol use disorder symptoms	7 (4–11)
Previously received care for alcohol use disorder	12
Alcoholics anonymous	4
Specialist treatment	3
Doctor or therapist	10
Previously used health apps	14
Primary care team member characteristics (n = 9)	n or M (range)
Age in years
Less than 39 years	6
40–49 years	2
50–59 years	1
Gender
Male	1
Female	8
Role on primary care team
Primary care provider	2
Social worker	5
Medical assistant	1
Community resource specialist	1
Years in practice	10 (1–25)

Note. AUDIT-C = Alcohol Use Disorders Identification Test-Consumption version.

Patient Perspectives

Five key themes emerged across patient participants (Table 2). Generally, these themes relate to factors that influence a patient's engagement and comfort in using an app for AUD in primary care. Supplemental Appendix D contains additional representative quotes.

Table 2.

Patient sample: Themes and representative quotes regarding the integration of alcohol apps in primary care.

Patient sample: Themes and representative quote(s) regarding the integration of alcohol apps in primary care^a
Patients seek a meaningful relationship with the clinician that guides their app use
If it's established that that is the person that you will be dealing with directly … this person has been assigned to help you, and then at that point it's like okay, cool, because then you understand that you are building a rapport with them as well. (P2, had severe AUD)
Patients want to see their healthcare team invested in the alcohol app
Literally being at the doctor's office, have the doctor saying if there was an issue with the drinking … Having the doctor literally be right there and be like ‘okay, hold on. Do you have your phone on you? Okay, pull up the thing—okay, cool.’ I feel like although I do think there is a lot on their plates, that having my doctor literally say download the app would be very impactful. (P10, had moderate AUD)
Patients want to be held accountable to using the app and addressing their drinking
I would be able to follow through better with having someone say, “I notice you haven’t used this part very much, and from the things you’ve been saying in this other section, this might be more useful for you. Are you having a problem with it? Why have you not been using it?” But in a less accusing kind of way (laughs). Why are you not using this! But I feel like the most efficient, to be able to help the most amount of people, is best, but so is accessibility and feeling achievement. (P12, has severe AUD)
Patients want the app and interactions with their healthcare team to be convenient
Not having to set a time to respond to something I think is important. That's again the benefits of texting, to be able to respond on your own schedule … I was thinking, why don’t they build that into the app kind of a thing, but text is so fast, it pops up right there, and you can respond without wading through the different levels down into opening the app and then the message center and that kind of a thing. (P14, has moderate AUD)
Patients value privacy regarding use of apps for alcohol
I think with something like this, it wouldn’t necessarily be something that I would want written, traceable or whatever. It's like it is somewhat sensitive information, talking about my drinking, naturally a bit sensitive. (P11, has severe AUD)

See Supplemental Appendix D for additional representative quotes.

Patients Seek a Meaningful Relationship With a Clinician Who Guides Their app use

Nearly all patients expressed the desire for a designated healthcare team member to guide their app use while they tried to address their drinking. They did not want to be sent home with an app only to work on it alone. Patients said it would be ideal for this designated person to be someone they have already established a relationship with. However, some expressed that they were open to working with a new person if they were “assigned to help you” (P2). A desire for consistency in this relationship stemmed from multiple factors. Patients wanted this person to “understand your history” (P3) and to be someone with whom they felt comfortable expressing themselves. There were also practical reasons for desiring a consistent person. Patients did not want to “rewrite the wheel every time if you talk to someone different” (P9).

Patients Want to see Their Healthcare Team Invested in the Alcohol app

Patients described that seeing their healthcare team place investment in an alcohol app would increase their engagement with the app and yield better outcomes. They noted specific healthcare team member behaviors that would demonstrate their investment, such as doctors telling patients to download an app, linking patients to team member who could help, or following up with patients about their app use. In contrast, providing an app and never “reaching out” would suggest a lack of investment (P5). Having an invested healthcare team would be “very beneficial” (P2), “very impactful” (P10), and would “breed trust” (P10) and make care “feel like a personalized treatment” (P7).

Patients Want to be Held Accountable to Using the app and Addressing Their Drinking

Patients said they would desire help from their healthcare team to stay “accountable” (P17) during their course of treatment with an alcohol app. Accountability was described as “having somebody to kind of wrangle you back in to keep it on track” (P2). Patients described that a range of interactions, particularly those involving outreach to the patient, would promote accountability. Having a healthcare team member review any self-reported data submitted through the app, such as daily drinking data, would help a patient be honest with oneself and their care team about their drinking. They also described that these accountability-promoting interactions should be executed with tact. Too much outreach or guidance could seem “overbearing” (P2) or “aggressive” (P4), and they would not want to feel “nagged” (P2). They also said that having this outreach done by someone on the care team with whom they had interacted before would feel “authentic” (P12).

Patients Want the app and Interactions With Their Healthcare Team to be Convenient

Patients described that interactions with their healthcare team and an alcohol app should be convenient and seamlessly integrated. Any care that is incident to the app should fit into their overall healthcare experience and their day-to-day life. Although most preferred to have an app introduced during a routine in-person healthcare visit, they wanted follow-up care to occur over messaging or telephone. One patient said that “blending” follow-up into existing visits “would probably be the most efficient way to do it” (P11). They also noted that a poorly designed app could potentially disrupt the flow of their healthcare visit, which could be avoided if the app was intuitive and easy to use. They did not want technical oversights to generate additional tasks (e.g., check-ins) or to disrupt time with their healthcare team.

Patients Value Privacy Regarding the use of Apps for AUD

Patients voiced privacy concerns in two general ways: having data about drinking or alcohol-related diagnoses stored digitally, and having it known by others, including healthcare system staff. Patients wanted assurance that their data would be secured and protected. Patients had mixed perspectives on whether information about their drinking should be shared at all. Some said that they could see the benefit of their healthcare provider having their drinking data. A few patients were more guarded and wanted control over who gets their drinking data and expressed ambivalence about allowing their healthcare providers access to this information. One patient said, “my docs are going to see it? Is someone else's doctor going to see it? It probably wouldn’t be a huge deterrent to me either way, but I would want to know” (P1).

Clinician Perspectives

Three major themes emerged from clinician interviews; these themes conveyed perspectives on encouraging app use by tailoring workflows to the patient and ensuring the feasibility of workflows for offering apps in primary care by addressing barriers and providing implementation support. An exemplary quote for each theme is presented in Table 3. Supplemental Appendix E contains additional representative quotes.

Table 3.

Clinician sample: Themes and representative quotes regarding the integration of alcohol treatment apps in primary care.

Clinician sample: Themes and representative quotes regarding the integration of alcohol treatment apps in primary care^a
Workflows to support apps in primary care should be tailored to the patient
It would also depend again on like the tech familiarity of the patient. Some people do respond better to talking, so the phone or even in person, but some people, it's going to be really hard … Maybe there could be options for different people because it seems like different people would do better with different follow up. I could see each of these being best for like a certain type of person. (C1, PCP)
Barriers to using apps in primary care: Staffing, complexity, and patient circumstances
Staffing
Being a PCP that feels strapped for time a lot, it's really great to be able to hand something off and say the social worker that I trust and kind of know could talk to her about that. I think that would be ideal, but realistically, yeah, there's some limitations. We don’t have unlimited social workers and they’re not just hanging out all the time to be ready to talk to somebody. (C1, PCP)
Intervention complexity
I think it's hard to lead and follow and track too much of this, and that makes it more complex. (C5, Social Worker)
Patient circumstances
I don’t think it should be required, like this has to be in person or something, because some people aren’t going to do that. They have too high of a copay or whatever it is, or they work business hours and they can’t come in. So I don’t think that should be like a requirement. (C1, PCP)
Appropriate staffing, simplified workflows, clinician education, and materials/tools are needed to support the implementation of apps in primary care
Appropriate staffing
If a health coach can do that and move them through it … why did I spend so long getting my license if I’m just going to do what the computer tells me? … A health coach who's an MA who is trained in health coaching and habit changing, they would be able to do it with clinical support of an LICSW … (C7, Social Worker)
Simplified workflows
If I could just say hey, here's this app, it's free … It's just a matter of making it easy, making sure we’re aware of what the value is. (C5, Social Worker)
Clinician education
I would need to know I guess the science behind it, like is there proof that it does help? I think that was one of the questions when I first—do doctor recommend it, or is it useful? Is it going to be beneficial? Like hey, studies show that this does help, then definitely, I think that's the biggest thing. (C4, Community Resource Specialist)
Materials/tools to help clinicians feel more competent in offering apps
So it would be important to probably have like a demo phone or video or something that would help—in this picture with the social worker, it looks like she's got a phone, so if there was some sort of demo phone, I guess. Or computer. I think that would be kind of the thing that would help. (C1, PCP)

Note. PCP = primary care provider.

See Supplemental Appendix E for additional representative quotes.

Workflows to Support Apps in Primary Care Should be Tailored to the Patient

Clinicians described that the approach for offering a digital intervention in primary care would need to be flexible to accommodate differences across patients. One clinician said, “I could see each of these [workflow approaches] being best for like a certain type of person” (C1, PCP).

Clinicians described that several patient characteristics could inform how to accommodate differences across patients. First, a patient's own availability and schedule could influence the clinical workflow. For instance, while most clinician participants preferred an approach where the PCP introduces the patient to an integrated mental health specialist who introduces the app and provides follow-up, several also said alternative approaches would be required for patients who would not wait for a social worker to become available on the same day. Second, the amount of app-related guidance offered by the clinician would “depend again on like the tech familiarity of the patient” (C1, PCP). Patients who were not tech savvy might need extensive guidance regarding installation, setup, and familiarization with the app. Third, clinical factors related to substance use would necessitate tailoring. Patients with more severe forms of unhealthy alcohol use might need more rigorous follow-up, whereas patients with mild unhealthy alcohol use might need less follow-up. “If it's just somebody who's wanting to get education about okay, why is three drinks a day maybe too much? But if they were really like struggling and this is a problem, I definitely think the more interaction the better” (C1, PCP). Similarly, clinicians thought a patient's readiness to address their drinking could impact their receptivity to conversations about an app-based treatment. For patients unlikely to speak with their healthcare team, patient-driven enrollment techniques such as a flyer with an app download link might work best, allowing the patient to seek help on their own. In summary, clinicians generally thought any workflow would need to be tailored to do “whatever really works for the patient” (C9, social worker).

Barriers to Using Apps in Primary Care: Staffing, Complexity, and Patient Circumstances

Clinicians alluded to barriers to offering alcohol apps, which fell into categories of staffing, workflow complexity, and patient-related barriers. Several clinicians said that current staffing gaps at their clinic could make it difficult to implement effective workflows for offering apps, “literally we have way too many jobs… we do all of it, so it's sped up in a way that isn’t conducive to doing the work in the way that is the most effective. We figure it out as we go” (C7, social worker). For several clinicians, despite preferring workflows that involved clinician support, staffing levels at their clinic led them to say that apps would ultimately need to be implemented in a way that requires no clinician support.

Clinicians also perceived that workflow processes for using apps with patients, particularly the setup process, involve a high level of complexity. One clinician who had experience with apps described the difficulty of making time to setup up a user account on an app vendor's website. Another clinician felt that it would be difficult to assist a patient with the setup process during a phone visit. This clinician said, “This scenario can be really complicated and have all these different variations, and it can be hard to explain that over the phone or to show people things” (C5, social worker).

Clinicians also described that some patients may face a variety of barriers to engaging in workflows that require in-person visits. Travel time to appointments was a barrier; “…with traffic it's not easy to get to [our clinic], and then that deters patients from wanting to get help” (C2, social worker). Another clinician said, “if they can’t come in because of childcare or work, whatever, you do what you can do” (C9, social worker). Relatedly, one clinician pointed out that costs could be higher for in-person visits, “some people aren’t going to [visit in person]. They have too high of a copay …” (C1, PCP).

Appropriate Staffing, Simplified Workflows, Clinician Education, and Materials/Tools are Needed to Support the Implementation of Apps in Primary Care

Clinicians offered a variety of ideas about additional support that would be necessary to offer apps in primary care and overcome anticipated barriers. These supports included appropriate staffing, simplified workflows, education and knowledge, and tools to help them feel more competent in offering apps.

Several clinicians suggested that additional staffing, particularly staffing other than PCPs, was necessary to optimize workflows. Several clinicians described that medical assistants who have specialized roles as health coaches could execute many aspects of the workflow that do not involve clinical decisions about substance use. One participant noted that an approach where social workers enroll patients, and medical assistants provide technical assistance and conduct check-ins, would allow clinicians to practice at the top of their licensure.

Clinicians also wanted a workflow that was simple with fewer steps. Regarding the workflow design, one clinician said, “If I could just say hey, here's this app, it's free … It's just a matter of making it easy, making sure we’re aware of what the value is” (C5, social worker). For PCPs, a simplified workflow meant knowing that they could “hand something off” to a trusted colleague who would set up the app with the patient and provide follow-up (C1, PCP).

They also described that a requisite level of knowledge would be necessary to make them feel comfortable to offer apps. This included scientific information about the effectiveness of apps, guidelines about patient eligibility criteria, and a general sense of knowing what the app includes, “I have to feel like I know something about it to be able to make a recommendation” (C3, PCP).

Additionally, clinicians would also want materials on hand to provide to the patient, which could assist the patient and make the clinician feel more confident in recommending the app. Examples of materials included informational handouts and videos that “give[s] them the basics” (C3, PCP). One clinician said, “if there's some sort of little video they could watch or something on a laptop that would have some info about the success of other people's experiences with the app, benefits of it, maybe a little brief overview with pictures on what it looks like, the app and stuff like that” (C1, PCP). Clinicians also wanted to have personal stories on hand about other patients who used and benefitted from the app. Finally, clinicians mentioned having devices on hand to demo the app for the patient.

Synthesis of Preferred Workflows for Integrating Apps for AUD in Primary Care

A synthesis of workflow-specific excerpts from the patient and clinician interviews yielded several basic principles for designing various workflow activities (Table 4). Furthermore, considering these excerpts and the number of participants who preferred each storyboard scenario (Supplemental Appendix F), the following sections describe a general framework for designing workflow approaches that are consistent with patient and clinician preferences.

Table 4.

Principles for designing workflow activities to integrate alcohol treatment apps in primary care

Patient perspectives	Clinician perspectives
Learning about the treatment app
A clinician, ideally the PCP, should introduce the app during visits where alcohol use is discussed.The clinician should have the knowledge and expertise to give advice about health and drinkingThe clinician should have adequate time to assist the patient and develop a personal connection.	Apps should be introduced during routine office visits.Introducing apps during in-person visits may “increase chances of follow-through” by patients.With adequate staffing, mental health specialists on the primary care team or medical assistants can be the point-person for the apps.PCPs should have enough knowledge about the app to “step-in” as needed.
Deciding whether to use an app
Patients need information about: Personal data the app will collect, who can access it, and security safeguards Ease of use of the app Time required to use the app Evidence regarding the app's use Expected health benefits The “ultimate end goal” of using the app	Clinicians need information about:
	Patient eligibility criteria Cost to the patient Research evidence The app's intervention content Simple app usage reports
Setting up the app
A clinician guides the patient in setting up the app during a visit where alcohol use was discussed, “why not just do it, take care of it right then.”Use the setup process for rapport building.	If time allows, help the patient setup the app “while you have their attention” because “if you watch them put it into their phone, they’re more likely to use it …”Patients who can setup the app on their own should be provided with a tech support contact.
Getting a walkthrough of the app
The clinician should provide a summary of the app's key feature and goals, “Like how do I track my drinking? … I could at least like know what I’m aiming for…”Ideally, the app should be intuitive and easy to use so the walkthrough can focus exclusively on content.	Aspects of the app walk-through can be automated or supported by technology such as video or a demo device with the app.
Getting tailored recommendations about how to use the app
Clinicians should recommend app features that align with each specific patient's needs or problems.“Personalized advice” would help improve the patient-clinician relationship, “it kind of breeds trust with the person you’re talking to.”Tailoring should aim to make the patient's use of the app more efficient, for example, directing them to pertinent content.	App recommendations may need to be tailored to a patient's severity of alcohol use.“Everything should be tailored to the patient,” but PCPs may find it difficult to make tailored recommendations unless the app was in use long term.
Making plans and conducting follow-up
Establish a plan for app use and follow-up at the outset to provide “some sense of structure and accountability,” improve clarity, and increase potential for engagement.Conducting follow-up via text message, the app, or a secure patient portal could increase convenience, but phone and in-person follow-up may be more effective.Some patients may want to use the app without follow-up.	Schedule a follow-up phone visit, reserving in-person follow-up in situations when it can be “tacked it onto another scheduled visit.”Follow-up solely over secure messages may be tempting, but patients often do not check messages.
Getting technical support
The app should be intuitive and user-friendly, negating the need for technical help.Support should be available via online materials, text, email, and phone.	Establish a contact for technical support.Instructional materials including videos and training should be available.
Paying for the app and associated care
Alcohol apps should be considered part of “preventative care” and covered by their health plan, especially if recommended by their care team.To avoid visit copays, app-related follow-up should be conducted in existing in-person visits, telephonically, or over secure message.	App-related costs and co-pays would be a barrier and should be avoided.

Note. PCP = primary care provider.

Candidate Workflow Approaches

Patient-Preferred Workflows

Most patients preferred to be introduced to a treatment app during a visit where alcohol use was discussed by a healthcare team member whom they trust. To the extent it was needed, most patients preferred assistance with setup and getting started at that same visit, with a plan for regular follow-up. Most patients preferred goals to be set for app usage, and with follow-up communication occurring primarily via messaging through a secure portal, conducting check-ins over the phone when needed. Most patients preferred any in-person follow-up, if it was necessary, to be bundled into existing visits. They preferred technical support to be available over the phone. Figure 1 illustrates a generalized representation of this patient-preferred workflow.

Figure 1.

A generalized representation of offering apps for alcohol use disorders in primary care.

Clinician-Preferred Workflows

Clinician perspectives about the ideal approach for integrating apps into primary care were notably consistent. All clinicians preferred workflows where alcohol apps were introduced to patients during a routine office visit, as opposed to workflows that included unsolicited outreach to patients who might benefit from apps, or workflows where patients signed up for apps themselves (e.g., pamphlets or websites). Introducing apps during routine visits would increase chances of follow-through by patients. While most clinicians described benefits of having PCPs initiate and follow patients’ use of apps, several acknowledged that PCPs probably would not have enough time to do this on a routine basis but should have the knowledge to step in as needed. Instead, participants preferred to have PCPs involve a mental health specialist to lead app delivery. Regarding follow-up, most clinicians preferred to conduct app-related follow-up over the phone, reserving in-person follow-up in situations when it can be added to another scheduled visit, and reserving secure messages for patients who regularly check them.

Discussion

By employing methods of user-centered design to elicit preferences of primary care patients and clinicians, this study identified several design principles for creating workflows and alleviating barriers to the use of apps for AUDs in primary care. The need for knowledge about feasible approaches for integrating apps into healthcare settings has been identified in numerous implementation and effectiveness studies (Graham, Greene, Powell, et al., 2020; Mares et al., 2016; Quanbeck et al., 2018; Thies et al., 2017). Whereas trials to date have obviated workflow barriers by involving researchers to deliver apps (Foley et al., 2016; Graham, Greene, Kwasny, et al., 2020; Graham, Greene, Powell, et al., 2020; Mares et al., 2016; Quanbeck et al., 2018), this study focused on improving knowledge about the design of approaches for embedding apps into real-world care in healthcare systems.

There were important areas of convergence between patient and provider preferences. Both patient and clinician samples described that workflows must be designed to minimize complexity. Patients in this study desired workflows that would seamlessly integrate apps into their primary care experience and day-to-day life. They emphasized that apps themselves should be intuitive and free of technical glitches to avoid interfering with their time with a clinician. Like in other studies (Lattie et al., 2020), clinicians advocated for workflows that were simple and required few steps to initiate patients onto an app-based treatment. Workflow processes would need to accommodate clinicians’ busy schedules and the various telecommunications mediums over which visits take place. Similarly, a case study of a specialty mental health “digital clinic” described the use of designated staff to provide app setup and technical support assistance so that clinical sessions would not be encumbered by these tasks (Rodriguez-Villa et al., 2020).

Patient and provider preferences also converged when expressing beliefs that patients would gain the most benefit if alcohol apps were offered and guided by a clinician, as opposed to making apps available to patients in the absence of clinician involvement. Both samples felt clinician guidance would help patients stay engaged and accountable to using apps, increasing the chances that patients would follow through with treatment recommendations. Relatedly, clinicians emphasized that they would need to tailor workflows on a case-by-case basis. For instance, clinicians felt that patients who have lower readiness to change or very severe AUDs would need more rigorous follow-up

To a large extent, these convergent findings suggest that researchers and healthcare systems should consider models of implementation whereby alcohol apps are used to supplement or enhance alcohol-related care that is provided by a clinician. As described in the implementation science and informatics literature, an “adjunctive” model of treatment involves clinician guidance to help patients use apps to add treatment activities and content for patients to engage with outside of visits (Hermes et al., 2019; Muñoz, 2017). This literature describes a continuum where adjunctive models of app-based care are placed on one end of the spectrum, and “fully automated” app-based care is on the other end. Fully automated models provide patients with access to apps with little to no guidance from a patient's healthcare team. We note that while findings overall suggested that an adjunctive model of care may be the most preferred implementation approach, preferences across participants were varied. Therefore, multiple pathways into app-based care may need to be offered concurrently to accommodate varying individual-level preferences and constraints within clinics. A case study of depression and anxiety app implementation in a US healthcare system reported success with providing multiple pathways to enroll patients in apps, including patient self-referral through a web-based healthcare portal and clinician-initiated electronic health record referral (Mordecai et al., 2021). However, two effectiveness trials reported less success in getting clinicians to submit referrals to apps via electronic health records (Graham, Greene, Kwasny, et al., 2020; Thies et al., 2017).

Given staffing shortages and limited time available during primary care visits, which are common constraints in primary care (Chernof et al., 1999; Jaen et al., 1994; Rost et al., 2000), clinicians indicated that staffing models could be altered to accommodate workflows for delivering apps. Specifically, they identified that workflow processes that do not involve clinical decision making, such as app account setup and periodic check-ins with patients, could be delegated to staff such as medical assistants who are trained to be health coaches. Indeed, primary care practices often use medical assistants to reduce clinician burden by reorganizing activities that do not require higher-level expertise (Bodenheimer et al., 2014; Powell et al., 2015). Such a solution could also reduce implementation costs and maximize sustainability, as all team members could focus on clinical duties that are fitting to their level of their licensure. Perhaps, such a model of app delivery could be a natural extension to team-based care models, where multiple clinicians on a team collaborate to care for the diverse needs of a given patient population (Schottenfeld et al., 2016). Given the preferences of patients and providers, studying implementation of app-based treatment in a collaborative care model for addressing AUD in primary care may be ideal.

We also note that participants expressed privacy concerns and wanted information how their data would be shared. This study and others underscore the importance of communicating risks and safeguards against accidental or malicious disclosure of private information, perhaps in clinician training and patient education materials (Lapham et al., 2012; Lattie et al., 2020).

This study may also have broader implications for implementation science and practice. Clinical workflow design and user-centered research methods could be particularly germane to the study of the implementation of digital interventions (Graham, Lattie, et al., 2020; Hermes et al., 2019), which require steps and systems that are relatively uncommon in traditional healthcare. More generally, some have described that many user-centered design methods could enhance the impact of implementation research by integrating techniques such as prototyping and engaging in iterative problem solving with end-users (Dopp et al., 2019, 2020). Likewise, implementation scientists have argued that clinical workflow design, or the practice of designing the series of steps and systems needed to accomplish a new clinical task, should be formally recognized as a discrete implementation strategy (Perry et al., 2019; Powell et al., 2012, 2015). Indeed, this study demonstrated that the use of these methods generated concrete recommendations and potentially generalizable principles regarding digital intervention workflow design and implementation.

For healthcare leaders or researchers seeking to use these findings to implement and/or study app-based treatments for AUDs, we recommend several additional design steps that are in line with the philosophy and procedures of user-centered design (International Organization for Standardization, 2019). First, the leader or researcher would engage local stakeholders from a target healthcare system to identify which themes and concrete suggestions from this study are applicable to the implementation context and the technical needs of the digital interventions being implemented. Next, this team would create workflow protocols and implementation procedures and establish an implementation plan (Dopp et al., 2019, 2020; Perry et al., 2019). This plan would specify quantitative outcome measures such as workflow errors and time needed to enroll patients, which would be tracked during deployment to inform further adjustments to workflows (Hermes et al., 2019). Given that each primary care practice will have different characteristics and needs, tailoring and adjustment should be expected, and any adaptations to implementation should be rigorously documented (Miller et al., 2021) and disseminated to further build knowledge about design and implementation. Furthermore, this study was conducted before COVID-19. Workflows may need tailoring to offer apps during virtual healthcare appointments and to reduce burden on clinics to accommodate additional mental health staffing shortages.

Limitations

This study is not without limitations, specifically its modest sample size. While it was a strength that we elicited perspectives of both patients and clinicians, only nine clinicians were interviewed. Additionally, perspectives about implementation and workflows in this study drew from participants’ past experiences, but none had previously used alcohol apps in primary care. Providing patients and clinicians with demo accounts and/or a video walkthrough of an alcohol app, and visual storyboards depicting potential workflow designs may have minimized this limitation. Participants were from five clinics of an integrated delivery system in the Pacific Northwest region of the United States; thus, results might not translate to non-integrated healthcare systems or geographic areas. Additionally, half of patient participants had college degrees, and most previously used apps, so findings may not generalize into settings where patients have lower education or digital literacy.

Conclusions

The perspectives of primary care patients with AUDs and primary care clinicians suggest that an optimal implementation of apps in primary care should involve clinician support, but multiple pathways into app-based care should be considered to accommodate implementation barriers and individual preferences. This study illustrates how patients and clinicians can be involved to improve clinical workflow design and identify determinants of implementation success, potentially increasing the likelihood of successful implementation and care outcomes. Future research will be needed to evaluate real-world feasibility and effectiveness of workflow and implementation strategies.

Supplemental Material

sj-docx-1-irp-10.1177_26334895221135264 - Supplemental material for Approaches for implementing digital interventions for alcohol use disorders in primary care: A qualitative, user-centered design study

Supplemental material, sj-docx-1-irp-10.1177_26334895221135264 for Approaches for implementing digital interventions for alcohol use disorders in primary care: A qualitative, user-centered design study by Joseph E. Glass, Brooks Tiffany, Theresa E. Matson, Catherine Lim, Gabrielle Gundersen, Kilian Kimbel, Andrea L. Hartzler, Geoffrey M. Curran, Angela Garza McWethy, Ryan M. Caldeiro and Katharine A. Bradley in Implementation Research and Practice

Footnotes

Declaration of Conflicting Interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Several authors (JEG, RMC, KAB) are investigators on a trial funded by the National Institute on Drug Abuse (R01DA047954; PI: Glass), which evaluates the implementation of reSET and reSET-O, which are digital therapeutics for substance use disorder marketed by Pear Therapeutics, Inc. Pear Therapeutics Inc provided digital therapeutic prescriptions at no cost to Kaiser Permanente Washington during a quality improvement pilot study that preceded the implementation trial. JEG is a coinvestigator on a Small Business Innovation Award funded awarded to Pear Therapeutics, Inc by the National Institute on Drug Abuse, which evaluates potential improvements to reSET-O (R44DA042652). This study is informing our research on the implementation of digital therapeutics, but Pear Therapeutics, Inc was not involved in this study and does not provide funding to the authors

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Research reported in this publication was supported by the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health under Award Number K01AA023859 (JEG) and K24AA022128 (KAB), and the Translational Research Institute through the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number UL1TR003107 (GMC). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Trial Registration

Not Applicable.

ORCID iDs

Joseph E. Glass

Gabrielle Gundersen

Supplemental Material

Supplemental material for this article is available online.

References

Auerbach

A. D.

(2019). Evaluating digital health tools-prospective, experimental, and real world. JAMA Internal Medicine, 179(6), 840–841. https://doi.org/10.1001/jamainternmed.2018.7229

Barry

C. L.

Epstein

A. J.

Fiellin

D. A.

Fraenkel

Busch

S. H.

(2016). Estimating demand for primary care-based treatment for substance and alcohol use disorders. Addiction, 111(8), 1376–1384. https://doi.org/10.1111/add.13364

Bernard

H. R.

Ryan

G. W.

(2002). Research methods in anthropology: Qualitative and quantitative methods. AltaMira Press.

Beyer

Holtzblatt

(1998). Contextual design. Morgan Kaufman.

Bickel

W. K.

Marsch

L. A.

Buchhalter

A. R.

Badger

G. J.

(2008). Computerized behavior therapy for opioid-dependent outpatients: A randomized controlled trial. Experimental and Clinical Psychopharmacology, 16(2), 132–143. https://doi.org/10.1037/1064-1297.16.2.132

Bodenheimer

Willard-Grace

Ghorob

(2014). Expanding the roles of medical assistants: Who does what in primary care? JAMA Internal Medicine, 174(7), 1025–1026. https://doi.org/10.1001/jamainternmed.2014.1319

Campbell

A. N.

Nunes

E. V.

Matthews

A. G.

Stitzer

Miele

G. M.

Polsky

Turrigiano

Walters

McClure

E. A.

Kyle

T. L.

Wahle

Van Veldhuisen

Goldman

Babcock

Stabile

P. Q.

Winhusen

Ghitza

U. E.

(2014). Internet-delivered treatment for substance abuse: A multisite randomized controlled trial. American Journal of Psychiatry, 171(6), 683–690. https://doi.org/10.1176/appi.ajp.2014.13081055

Carayon

(2009). The balance theory and the work system model … twenty years later. International Journal of Human–Computer Interaction, 25(5), 313–327. https://doi.org/10.1080/10447310902864928

Center for Disease Control and Prevention. (2015a). Alcohol and public health: Data, trends, and maps. Centers for Disease Control and Prevention. Retrieved June 12, 2021, from http://www.cdc.gov/alcohol/quickstats/binge_drinking.htm

10.

Center for Disease Control and Prevention. (2015b). Fact sheets—Binge drinking. Center for Disease Control and Prevention. Retrieved June 12, 2021, from http://www.cdc.gov/alcohol/quickstats/binge_drinking.htm

11.

Chernof

B. A.

Sherman

S. E.

Lanto

A. B.

Lee

M. L.

Yano

E. M.

Rubenstein

L. V.

(1999). Health habit counseling amidst competing demands: Effects of patient health habits and visit characteristics. Medical Care, 37(8), 738–747. https://doi.org/10.1097/00005650-199908000-00004

12.

Christensen

D. R.

Landes

R. D.

Jackson

Marsch

L. A.

Mancino

M. J.

Chopra

M. P.

Bickel

W. K.

(2014). Adding an internet-delivered treatment to an efficacious treatment package for opioid dependence. Journal of Consulting and Clinical Psychology, 82(6), 964–972. https://doi.org/10.1037/a0037496

13.

Creswell

J. W.

(2013). Qualitative inquiry & research design: Choosing among five approaches (3rd ed.). Sage.

14.

Crowley

R. A.

Kirschner

Health & Public Policy Committee of the American College of Physicians. (2015). The integration of care for mental health, substance abuse, and other behavioral health conditions into primary care: Executive summary of an American college of physicians position paper. Annals of Internal Medicine, 163(4), 298–299. https://doi.org/10.7326/M15-0510

15.

Degenhardt

Hall

(2012). Extent of illicit drug use and dependence, and their contribution to the global burden of disease. The Lancet, 379(9810), 55–70. https://doi.org/10.1016/S0140-6736(11)61138-0

16.

Dopp

A. R.

Parisi

K. E.

Munson

S. A.

Lyon

A. R.

(2019). A glossary of user-centered design strategies for implementation experts. Translational Behavioral Medicine, 9(6), 1057–1064. https://doi.org/10.1093/tbm/iby119

17.

Dopp

A. R.

Parisi

K. E.

Munson

S. A.

Lyon

A. R.

(2020). Aligning implementation and user-centered design strategies to enhance the impact of health services: Results from a concept mapping study. Implementation Science Communications, 1, 17. https://doi.org/10.1186/s43058-020-00020-w

18.

Foley

Steinberg

Levine

Askew

Batch

B. C.

Puleo

E. M.

Svetkey

L. P.

Bosworth

H. B.

DeVries

Miranda

Bennett

G. G.

(2016). Track: A randomized controlled trial of a digital health obesity treatment intervention for medically vulnerable primary care patients. Contemporary Clinical Trials, 48, 12–20. https://doi.org/10.1016/j.cct.2016.03.006

19.

Ghitza

U. E.

Tai

(2014). Challenges and opportunities for integrating preventive substance-use-care services in primary care through the affordable care act. Journal of Health Care for the Poor and Underserved, 25(1 Suppl), 36–45. https://doi.org/10.1353/hpu.2014.0067

20.

Glass

J. E.

Bobb

J. F.

Lee

A. K.

Richards

J. E.

Lapham

G. T.

Ludman

Achtmeyer

Caldeiro

R. M.

Parrish

Williams

E. C.

Lozano

Bradley

K. A.

(2018). Study protocol: A cluster-randomized trial implementing sustained patient-centered alcohol-related care (SPARC trial). Implementation Science, 13(1), 108. https://doi.org/10.1186/s13012-018-0795-9

21.

Glass

J. E.

Matson

T. E.

Lim

Hartzler

A. L.

Kimbel

Lee

A. K.

Beatty

Parrish

Caldeiro

R. M.

Garza McWethy

Curran

G. M.

Bradley

K. A.

(2021). Approaches for implementing app-based digital treatments for drug use disorders into primary care: A qualitative, user-centered design study of patient perspectives. Journal of Medical Internet Research, 23(7), e25866. https://doi.org/10.2196/25866

22.

Glass

J. E.

McKay

J. R.

Gustafson

D. H.

Kornfield

Rathouz

P. J.

McTavish

F. M.

Atwood

A. K.

Isham

Quanbeck

Shah

(2017). Treatment seeking as a mechanism of change in a randomized controlled trial of a mobile health intervention to support recovery from alcohol use disorders. Journal of Substance Abuse Treatment, 77, 57–66. https://doi.org/10.1016/j.jsat.2017.03.011

23.

Glass

J. E.

Perron

B. E.

Ilgen

M. A.

Chermack

S. T.

Ratliff

Zivin

(2010). Prevalence and correlates of specialty substance use disorder treatment for department of veterans affairs healthcare system patients with high alcohol consumption. Drug and Alcohol Dependence, 112(1–2), 150–155. https://doi.org/10.1016/j.drugalcdep.2010.06.003

24.

Graham

A. K.

Greene

C. J.

Kwasny

M. J.

Kaiser

S. M.

Lieponis

Powell

Mohr

D. C.

(2020). Coached mobile app platform for the treatment of depression and anxiety among primary care patients: A randomized clinical trial. JAMA Psychiatry, 77(9), 906–914. https://doi.org/10.1001/jamapsychiatry.2020.1011

25.

Graham

A. K.

Greene

C. J.

Powell

Lieponis

Lunsford

Peralta

C. D.

Orr

L. C.

Kaiser

S. M.

Alam

Berhane

Kalan

Mohr

D. C.

(2020). Lessons learned from service design of a trial of a digital mental health service: Informing implementation in primary care clinics. Translational Behavioral Medicine, 10(3), 598–605. https://doi.org/10.1093/tbm/ibz140

26.

Graham

A. K.

Lattie

E. G.

Powell

B. J.

Lyon

A. R.

Smith

J. D.

Schueller

S. M.

Stadnick

N. A.

Brown

C. H.

Mohr

D. C.

(2020). Implementation strategies for digital mental health interventions in health care settings. American Psychologist, 75(8), 1080–1092. https://doi.org/10.1037/amp0000686

27.

Graneheim

U. H.

Lundman

(2004). Qualitative content analysis in nursing research: Concepts, procedures and measures to achieve trustworthiness. Nurse Education Today, 24(2), 105–112. https://doi.org/10.1016/j.nedt.2003.10.001

28.

Grant

B. F.

Goldstein

R. B.

Saha

T. D.

Chou

S. P.

Jung

Zhang

Pickering

R. P.

Ruan

W. J.

Smith

S. M.

Huang

Hasin

D. S.

(2015). Epidemiology of DSM-5 alcohol use disorder: Results from the national epidemiologic survey on alcohol and related conditions III. JAMA Psychiatry, 72(8), 757–766. https://doi.org/10.1001/jamapsychiatry.2015.0584

29.

Gustafson

D. H.

McTavish

F. M.

Chih

M.-Y.

Atwood

A. K.

Johnson

R. A.

Boyle

M. G.

Levy

M. S.

Driscoll

Chisholm

S. M.

Dillenburg

Isham

Shah

(2014). A smartphone application to support recovery from alcoholism: A randomized clinical trial. JAMA Psychiatry, 71(5), 566–572. https://doi.org/10.1001/jamapsychiatry.2013.4642

30.

Hermes

E. D.

Lyon

A. R.

Schueller

S. M.

Glass

J. E.

(2019). Measuring the implementation of behavioral intervention technologies: Recharacterization of established outcomes. Journal of Medical Internet Research, 21(1), e11752. https://doi.org/10.2196/11752

31.

Institute of Medicine. (2006). Improving the quality of health care for mental and substance-use conditions. National Academy Press.

32.

International Organization for Standardization. (2019). Ergonomics of human–system interaction—Part 210: Human-centred design for interactive systems. I.-. 2019. (2nd ed). Switzerland.

33.

Jaen

C. R.

Stange

K. C.

Nutting

P. A.

(1994). Competing demands of primary care: A model for the delivery of clinical preventive services. The Journal of Family Practice, 38(2), 166–171.

34.

Kaner

E. F.

Beyer

F. R.

Garnett

Crane

Brown

Muirhead

Redmore

O’Donnell

Newham

J. J.

de Vocht

Hickman

Brown

Maniatopoulos

Michie

(2017). Personalised digital interventions for reducing hazardous and harmful alcohol consumption in community-dwelling populations. Cochrane Database of Systematic Reviews, 9, CD011479. https://doi.org/10.1002/14651858.CD011479.pub2

35.

Kiluk

B. D.

Devore

K. A.

Buck

M. B.

Nich

Frankforter

T. L.

LaPaglia

D. M.

Yates

B. T.

Gordon

M. A.

Carroll

K. M.

(2016). Randomized trial of computerized cognitive behavioral therapy for alcohol use disorders: Efficacy as a virtual stand-alone and treatment add-on compared with standard outpatient treatment. Alcoholism: Clinical and Experimental Research, 40(9), 1991–2000. https://doi.org/10.1111/acer.13162

36.

Kiluk

B. D.

Ray

L. A.

Walthers

Bernstein

Tonigan

J. S.

Magill

(2019). Technology-delivered cognitive-behavioral interventions for alcohol use: A meta-analysis. Alcoholism: Clinical and Experimental Research, 43(11), 2285–2295. https://doi.org/10.1111/acer.14189

37.

King

(2004). Using templates in the thematic analysis of text. In Cassell

Symon

(Eds.), Essential guide to qualitative methods in organizational research (pp. 256–270). London: SAGE Publications.

38.

Knowles

S. E.

Lovell

Bower

Gilbody

Littlewood

Lester

(2015). Patient experience of computerised therapy for depression in primary care. BMJ Open, 5(11), e008581. https://doi.org/10.1136/bmjopen-2015-008581

39.

Lapham

G. T.

Hawkins

E. J.

Chavez

L. J.

Achtmeyer

C. E.

Williams

E. C.

Thomas

R. M.

Ludman

E. J.

Kypri

Hunt

S. C.

Bradley

K. A.

(2012). Feedback from recently returned veterans on an anonymous web-based brief alcohol intervention. Addiction Science & Clinical Practice, 7(1), 17. https://doi.org/10.1186/1940-0640-7-17

40.

Lattie

E. G.

Nicholas

Knapp

A. A.

Skerl

J. J.

Kaiser

S. M.

Mohr

D. C.

(2020). Opportunities for and tensions surrounding the use of technology-enabled mental health services in community mental health care. Administration and Policy in Mental Health and Mental Health Services Research, 47(1), 138–149. https://doi.org/10.1007/s10488-019-00979-2

41.

MacQueen

K. M.

McLellan

Kay

Milstein

(1998). Codebook development for team-based qualitative analysis. Field Methods, 10(2), 31–36. https://doi.org/10.1177/1525822X980100020301

42.

Mares

M. L.

Gustafson

D. H.

Glass

J. E.

Quanbeck

McDowell

McTavish

Atwood

A. K.

Marsch

L. A.

Thomas

Shah

Brown

Isham

Nealon

M. J.

Ward

(2016). Implementing an mHealth system for substance use disorders in primary care: A mixed methods study of clinicians’ initial expectations and first year experiences. BMC Medical Informatics and Decision Making, 16(1), 126. https://doi.org/10.1186/s12911-016-0365-5

43.

Marsch

L. A.

Guarino

Acosta

Aponte-Melendez

Cleland

Grabinski

Brady

Edwards

(2014). Web-based behavioral treatment for substance use disorders as a partial replacement of standard methadone maintenance treatment. Journal of Substance Abuse Treatment, 46(1), 43–51. https://doi.org/10.1016/j.jsat.2013.08.012

44.

Miller

C. J.

Barnett

M. L.

Baumann

A. A.

Gutner

C. A.

Wiltsey-Stirman

(2021). The FRAME-IS: A framework for documenting modifications to implementation strategies in healthcare. Implementation Science, 16(1), 36. https://doi.org/10.1186/s13012-021-01105-3

45.

Mohr

D. C.

Cuijpers

Lehman

(2011). Supportive accountability: A model for providing human support to enhance adherence to eHealth interventions. Journal of Medical Internet Research, 13(1), e30. https://doi.org/10.2196/jmir.1602

46.

Mohr

D. C.

Lyon

A. R.

Lattie

E. G.

Reddy

Schueller

S. M.

(2017). Accelerating digital mental health research from early design and creation to successful implementation and sustainment. Journal of Medical Internet Research, 19(5), e153. https://doi.org/10.2196/jmir.7725

47.

Mokdad

A. H.

Marks

J. S.

Stroup

D. F.

Gerberding

J. L.

(2004). Actual causes of death in the United States, 2000. JAMA, 291(10), 1238–1245. https://doi.org/10.1001/jama.291.10.1238

48.

Mordecai

Histon

Neuwirth

Heisler

W. S.

Kraft

Bang

Franchino

Taillac

Nixon

J. P.

(2021). How Kaiser Permanente created a mental health and wellness digital ecosystem. NEJM Catalyst, 2(1), 1–21. https://doi.org/10.1056/CAT.20.0295

49.

Muñoz

R. F.

(2017). The efficiency model of support and the creation of digital apothecaries. Clinical Psychology: Science and Practice, 24(1), 46–49. https://doi.org/10.1111/cpsp.12174

50.

National Council for Mental Wellbeing. (2018). Implementing care for alcohol & other drug use in medical settings: An extension of SBIRT. Retrieved June 12, 2021 from https://www.thenationalcouncil.org/wp-content/uploads/2020/08/021518_NCBH_ASPTReport-FINAL.pdf?daf=375ateTbd56

51.

National Institute on Alcohol Abuse and Alcoholism. (2007). Helping patients who drink too much: A clinician’s guide (updated 2005 edition). National Institutes of Health, U.S. Department of Health and Human Services. Retrieved June 12, 2021, from https://pubs.niaaa.nih.gov/publications/practitioner/cliniciansguide2005/guide.pdf

52.

O’Connor

Hanlon

O’Donnell

C. A.

Garcia

Glanville

Mair

F. S.

(2016). Understanding factors affecting patient and public engagement and recruitment to digital health interventions: A systematic review of qualitative studies. BMC Medical Informatics and Decision Making, 16(1), 120. https://doi.org/10.1186/s12911-016-0359-3

53.

Padgett

(2008). Qualitative methods in social work research (2nd ed.). Sage Publications.

54.

Perry

C. K.

Damschroder

L. J.

Hemler

J. R.

Woodson

T. T.

Ono

S. S.

Cohen

D. J.

(2019). Specifying and comparing implementation strategies across seven large implementation interventions: A practical application of theory. Implementation Science, 14(1), 32. https://doi.org/10.1186/s13012-019-0876-4

55.

Powell

B. J.

McMillen

J. C.

Proctor

E. K.

Carpenter

C. R.

Griffey

R. T.

Bunger

A. C.

Glass

J. E.

York

J. L.

(2012). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2), 123–157. https://doi.org/10.1177/1077558711430690

56.

Powell

B. J.

Waltz

T. J.

Chinman

M. J.

Damschroder

L. J.

Smith

J. L.

Matthieu

M. M.

Proctor

E. K.

Kirchner

J. E.

(2015). A refined compilation of implementation strategies: Results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10, 21. https://doi.org/10.1186/s13012-015-0209-1

57.

Quanbeck

Gustafson

D. H.

Marsch

L. A.

Chih

M. Y.

Kornfield

McTavish

Johnson

Brown

R. T.

Mares

M. L.

Shah

D. V.

(2018). Implementing a mobile health system to integrate the treatment of addiction into primary care: A hybrid implementation-effectiveness study. Journal of Medical Internet Research, 20(1), e37. https://doi.org/10.2196/jmir.8928

58.

Quanbeck

A. R.

Gustafson

D. H.

Marsch

L. A.

McTavish

Brown

R. T.

Mares

M.-L.

Johnson

Glass

J. E.

Atwood

A. K.

McDowell

(2014). Integrating addiction treatment into primary care using mobile health technology: Protocol for an implementation research study. Implementation Science, 9(1), 65. https://doi.org/10.1186/1748-5908-9-65

59.

Ramsey

A. T.

Satterfield

J. M.

Gerke

D. R.

Proctor

E. K.

(2019). Technology-based alcohol interventions in primary care: Systematic review. Journal of Medical Internet Research, 21(4), e10859. https://doi.org/10.2196/10859

60.

Rodriguez-Villa

Rauseo-Ricupero

Camacho

Wisniewski

Keshavan

Torous

(2020). The digital clinic: Implementing technology and augmenting care for mental health. General Hospital Psychiatry, 66, 59–66. https://doi.org/10.1016/j.genhosppsych.2020.06.009

61.

Rost

Nutting

Smith

Coyne

J. C.

Cooper-Patrick

Rubenstein

(2000). The role of competing demands in the treatment provided primary care patients with major depression. Archives of Family Medicine, 9(2), 150–154. https://doi.org/10.1001/archfami.9.2.150

62.

Saitz

Larson

M. J.

Labelle

Richardson

Samet

J. H.

(2008). The case for chronic disease management for addiction. Journal of Addiction Medicine, 2(2), 55–65. https://doi.org/10.1097/ADM.0b013e318166af74

63.

Saunders

Sim

Kingstone

Baker

Waterfield

Bartlam

Burroughs

Jinks

(2018). Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality & Quantity, 52(4), 1893–1907. https://doi.org/10.1007/s11135-017-0574-8

64.

Schottenfeld

Petersen

Peikes

Ricciardi

Burak

McNellis

Genevro

(2016). Creating patient-centered team-based primary care . Rockville: Agency for Healthcare Research and Quality.

65.

Sheehan

D. V.

Lecrubier

Harnett Sheehan

Janavs

Weiller

Keskiner

Schinka

Knapp

Sheehan

M. F.

Dunbar

G. C.

(1997). The validity of the Mini International Neuropsychiatric Interview (MINI) according to the SCID-P and its reliability. European Psychiatry, 12(5), 232–241. https://doi.org/10.1016/S0924-9338(97)83297-X

66.

Sheehan

D. V.

Lecrubier

Sheehan

K. H.

Amorim

Janavs

Weiller

Hergueta

Baker

Dunbar

G. C.

(1998). The Mini-International Neuropsychiatric Interview (M.I.N.I.): The development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. Journal of Clinical Psychiatry, 59(Suppl 20), 22–33.

67.

Sobell

L. C.

Agrawal

Sobell

M. B.

Leo

G. I.

Young

L. J.

Cunningham

J. A.

Simco

E. R.

(2003). Comparison of a quick drinking screen with the timeline followback for individuals with alcohol problems. Journal of Studies on Alcohol, 64(6), 858–861. https://doi.org/10.15288/jsa.2003.64.858

68.

SocioCultural Research Consultants. (2018). Dedoose (Version 8.0.35) [Computer software]. https://www.dedoose.com/

69.

Thies

Anderson

Cramer

(2017). Lack of adoption of a mobile app to support patient self-management of diabetes and hypertension in a federally qualified health center: Interview analysis of staff and patients in a failed randomized trial. JMIR Human Factors, 4(4), e24. https://doi.org/10.2196/humanfactors.7709

70.

Truong

K. N.

Hayes

G. R.

Abowd

G. D.

(2006). Storyboarding: An empirical determination of best practices and effective guidelines . In Proceedings of the 6th Conference on Designing Interactive Systems. DIS '06. University Park, PA, USA, June 26–28, 2006, ACM, New York, NY. https://doi.org/10.1145/1142405.1142410

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.29 MB