Abstract
Background
The “translation” of health-related scientific discoveries into clinical guidelines and everyday clinical practice can take years or decades, and many discoveries may never make it into daily practice.1,2 The advent of evidence-based guidelines in contrast to consensus-based guidelines has been a major advance. 3 Many of these discoveries and guidelines pertain to the wide spectrum of health conditions and ages encountered in primary care. However, discoveries and guidelines do not exist for all health conditions that are of priority to communities, and in some cases the evidence and guidelines might apply only to a limited group (such as certain ages, cultures, or ethnicities). Indeed, the original articles describing strategies for classifying evidence acknowledge the existence and need for observational evidence. 4 5 In these instances, a method is needed to develop community relevant evidence from patients and community members who have found their own pathways to success through perseverance or trial and error, instead of using an academic, top down, barrier-focused approach that ignores the resourcefulness, skills, and knowledge of patients and community assets.
We first became aware of these potential evidence limitations through the work of our long-standing partnership between the High Plains Research Network (HPRN) Community Advisory Council (CAC) and the University of Colorado Denver’s Department of Family Medicine (CU-DFM). The HPRN CAC had previously identified mental health as a priority issue in their region of eastern Colorado. To help focus their goals for patient-centered research and action, the group undertook a year-long process of learning about mental health from various experts who covered definitions and epidemiology of major conditions, integrated behavioral health, suicide prevention, and policy.
As explained by HPRN CAC member, Chris Bennett, We began studying mental health in eastern Colorado by looking at lots of research to improve mental health outcomes. Most of it revolved around implementing evidence-based guidelines for specific treatments for specific illnesses or system-level changes. As each expert talked, we routinely kept coming back to, “That’s awesome work, but not quite what we think our community needs.” We realized that we wanted to tackle the problem of “How do we help people overcome the barriers to access the mental healthcare system and support before they have a mental health crisis?” This was the question that wasn’t sufficiently answered in the research or evidence-based guidelines.
Maret Felzien, another member of the HPRN CAC, added, The existing guidelines and information didn’t introduce a way forward for us. We needed a way for us to work on improving and preventing mental health outcomes from a locally informed base. We needed a round peg for a round hole versus a square peg in a round hole.
Inspired by the experience of the HPRN C.A.C., the CU-DFM’s SNOCAP (State Networks Of Colorado Ambulatory Practices & Partners) practice-based research networks’ Community and Patient Advisory Councils and 2040 Partners for Health (2040) worked to fill this methods gap. We intended to address the methods gap for sourcing and translating community and practice-based evidence by identifying and testing the use of the new method described in this report. We call this method Appreciative Inquiry/Boot Camp Translation (AI/BCT). AI is a method that uncovers and learns from successful occurrences and conditions that foster success.6-8 In AI, participants are interviewed about their successes, and the qualitative analysis of these successes seeks to identify common themes that play an important role in the participants’ successes. BCT is a participatory method developed by the partnership between the HPRN CAC and CU-DFM to translate medical jargon into messages and materials that are locally relevant, actionable, and attainable. BCT has proven itself with multiple sets of guidelines and evidence as an effective, evidence-based method for translation of these into locally relevant messages and materials.9-13 The resulting materials and their dissemination have impacted colorectal cancer screening rates, asthma exacerbations, and delivery of self-management support among others.12,14,15 The joining of these methods into a new combined method called AI/BCT is designed to access local patient and community members’ successes and translate those random successes into evidence and interventions that will relevant and become more usual within a community. We obtained Methods Award funding from the Patient-Centered Research Outcomes Institute (ME-1303-5843) to pilot the AI/BCT method and identify aspects and lessons learned that would facilitate use of the method by other academic/patient partnerships.
The purpose of this article is to primarily describe the rationale for AI/BCT, while briefly describing the projects we used to pilot and evaluate the method with a discussion of important lessons learned from this initial formal evaluation of the AI/BCT method. We will describe the 5 topic areas for which “on the shelf” evidence was lacking, our AI/BCT approach to address the gap in evidence for each, an overview of the products that emerged from each, and the key learnings from our study. For those interested in a full description of our methods and their formative evaluation, we will present those in a subsequent paper.
Methods
We began our investigation of AI/BCT by working with our community partners to identify topic areas with which we could implement and evaluate AI/BCT. To identify the topics for AI/BCT projects, we used a Delphi method approach to reach consensus on the important health issues of priority within each community. 16 We supplemented this discussion with a review of the academic literature on the community-identified topics to determine need, feasibility, and current evidence base. This yielded the 5 topic areas in Table 1.
Health Topics Identified by Community Partners for Appreciative Inquiry/Boot Camp Translation (AI/BCT).
For each individual topic we worked with the associated community partners in Table 1 to identify the specific evidence gaps, design recruitment materials, design an AI interview guide, and recruit eligible participants. The study protocol was approved by the Colorado Multiple Institutional Review Board (COMIRB). As approved by COMIRB, verbal informed consent was obtained from each AI interview participant.
Successful access to mental health care provides an illustrative example of a topic, identified by our community partners, for which we found an evidence gap. Mental health care access is frequently conceptualized as being related to an individual’s ability to pay for services or having direct access to a mental health care provider. However, our communities were instead concerned with aspects of access related an individual’s recognition of the need for help and their subsequent and sustained access to help in the form of professional care. Think of this as crossing the boundary between individual recognition of symptoms and seeking and receipt of care. Indeed, the challenges associated with crossing this boundary were amply illustrated by the stories told by our interviewees. Successful access, diagnosis, and treatment also underlay the topic of sleep apnea. For the topic of chronic pain management, our community partners identified that evidence was lacking as to how individuals successfully gain access to, and navigate through, the multitude of potential treatment modalities. Finally, there was a desire to uncover the elements of successful primary care practice transformation to patient-centered care for practices that have experience in support programs but may still exhibit resistance to transformation.
The AI interviews were conducted by our study team staff, proceeding from the key question for each topic as shown in Table 2. The majority of the interviews were with one participant, and all interviews were recorded and transcribed for qualitative analysis to identify key themes. The qualitative analysis for each topic was led by 2 of our team members (JSH and TLH) in a grounded theory approach, using a combination of editing and immersion-crystallization styles.17,18 This consisted of iterative review of codes and themes by the entire study team, including members of our community partnerships.
Appreciative Inquiry/Boot Camp Translation (AI/BCT) Topics, AI Key Questions, Resulting Themes and BCT Products.
Following the AI analysis, the BCT phase began for each topic. Using a typical BCT process, we recruited a set of 11 to 14 BCT participants for each topic from the corresponding partner community. Two study team members facilitated each BCT with additional participation by one of the qualitative leads, who served in the role of an “interview data content expert,” much as a topic content expert participates in the usual BCT process. Beyond this variation for each topic, a typical BCT process was followed, and each topic had results of community relevant messages and materials.
Figure 1 summarizes the AI/BCT process. A detailed description of our methods will be presented in a subsequent paper.
Overview of the Appreciative Inquiry/Boot Camp Translation (AI/BCT) process.
Results
We found AI/BCT is a useful method to create locally relevant evidence for addressing important health issues where little, if any, evidence currently exists. We successfully conducted AI/BCT processes around our five community identified topics of interest. Table 2 summarizes for each topic the key AI interview question, the main themes that emerged from the AI interviews, and the final BCT messages and sample products. Each iteration of our AI/BCT process yielded AI findings and themes that were then translated by a BCT process into messages and materials that can be used by the community to address the health topic or concern.
Beyond the results of the messages and materials produced by each AI/BCT, we developed several important lessons about the application of the AI/BCT method: Researchers must (1) first ensure whether the topic is a good fit for AI, (2) maintain a focus on “what works” throughout all stages, (3) recruit one or more experienced qualitative analysts, (4) ensure adequate time and resources for the extensive AI/BCT process, and (5) present AI findings to BCT participants in the context of existing evidence and the local community and allow time for community partners to ask questions and request additional data analyses to be done (Figure 2) .
Key, practical learnings for Appreciative Inquiry/Boot Camp Translation (AI/BCT) implementation.
First, selection of topics suitable for AI is important; AI requires stories of success. Identifying priority health topics in communities should begin with an open-ended inquiry, which is then focused with the help of academic partners. As lists are refined with the Delphi method, 16 feasibility of the topics becomes an important consideration. Does the community believe finding stories of success will be difficult? If so, the topic may not work. Does the topic have an existing significant evidence base that is relevant to the community partners? If so, community-academic partnerships should opt to use the traditional BCT process alone. If neither of these apply, AI/BCT could be a good fit for the topic chosen. Scope is important to consider as well. Examples of priority topics that are beyond the scope of AI/BCT include large community issues such as making health insurance accessible and addressing pharmaceutical costs. Topics that are typically a good fit for AI are locally relevant; in other words, they are common issues in the community and a priority need in the minds of community members; and feasible for a community-level intervention.
Another important step to determine if a topic is a good fit for AI is whether success can be clearly defined. For example, the AI/BCT on chronic pain management began with the broad topic of chronic pain. After discussion, the team specified a focus on the management of pain. With help from community partners, we chose to define success as managing chronic pain in a way that asks, “Are you able to do most of what you want to do on most days of the week?”
Second, a focus on “what worked” for AI interviews is important. Interviewers should be trained in the AI approach. Focusing on “what worked” during interviews can be challenging, especially to researchers trained in problem-focused approaches. Utilizing 2 co-interviewers for data collection efforts can help ensure that interviews are complete and capture elements of success. Within the AI data collection component, we also found that individual interviews delivered more in-depth and robust stories of success than small groups. Group interviews are not recommended as obtaining individual stories of success was more challenging in a group format than individual interviews.
Third, selecting one or more experienced qualitative analysts for the AI data is crucial. The entire AI/BCT research team does not need to be skilled in AI data analysis. However, the researcher leading the analysis of the AI data should have advanced qualitative analysis skills. Coding, identifying common themes, and organizing results require an analyst that remains in the “what worked” frame of mind. Further, identifying themes and organizing data for BCT require an analytic lens that comes from a solid understanding of both AI and BCT.
Fourth, ensure adequate time and team resources for the process. Researchers using AI/BCT should allow a minimum of four months to collect, code, analyze, and organize the AI interview data. The time requirements should be factored into budgeting on the academic side and made transparent for community partners as well to avoid expectations of more rapid results. The analyst works with a team, particularly in the early stages of reviewing transcripts and creating a coding system. This coding team should include at least one interviewer, if the analyst did not participate in interviews. Coding analyst teams should incorporate members with specific subject matter expertise to shed light on terms or procedures, if applicable. For example, the Practice Transformation AI/BCT coding team included a physician to provide insights and clarifications that only a practicing physician could offer.
Finally, take care in how the AI data is presented during the BCT stage. AI findings are incorporated into the BCT process as the new, locally generated evidence base for the selected topic. We tested and confirmed the value of a 2-part educational presentation at the BCT kick-off meeting, which is slightly altered from traditional BCT. First, a topic expert provides a presentation on what is known about the topic. This is followed by a presentation of the AI process and results, including themes of how successes in that particular community have been achieved. The BCT facilitation team and AI data analyst need to be able to articulate both the local relevance of the AI data and the scientific rigor used to collect and analyze the data. It is also important at this stage to allow the community partners to request additional data be analyzed for review and presentation at a later meeting. This came into play during the chronic pain management AI/BCT when BCT participants wanted to know more about why more women were interviewed than men—was this due to convenience sampling or that women were more often likely to tell their story?
Discussion
We conceived AI/BCT as a method to address health concerns that are of importance to communities, but for which evidence of how to address the topic is either lacking or a poor fit due to the unique circumstances of a particular community. The contextual relevance of evidence and solutions to community health concerns is critical. As discussed in the introduction, our experience with one of our communities’ concerns regarding access to mental health care led us to devise a method that could leverage isolated successes of community members to create approaches that could be applied across a community.
AI is just beginning to be applied to the health arena. However, much of the focus of this work is on the organization of health care.7,8,19-24 We believed that AI had potential as a method to uncover local community members’ and patients’ solutions to their health concerns, in effect generating patient-centered evidence for what works. In our study we successfully used AI to generate stories of success from community members struggling with health concerns. These formed the basis of themes that were successfully translated by BCT participants into interventions that can be broadly used in their communities to address the health topics and concerns.
Key understandings that emerged from our study were the following: AI/BCT is not a method for any health topic. Where evidence exists for a particular health topic and simply needs translation into locally relevant messages and materials, the standard BCT process is very appropriate. For broader topics that may require policy solutions, AI/BCT is not a good fit. AI requires a focus on what works during interviews. Qualitative analysts must understand the AI approach and focus on successes. The AI part of the process adds significantly to the time and resources required. Finally, AI findings must be presented to BCT participants along with additional context of what is already known about the health topic. This helps weave together current knowledge and locally relevant evidence to address each topic. If a trained BCT facilitator is not available, we recommend interested academic/community partnerships obtain training. We hold trainings regularly for that purpose.
AI/BCT addresses a methods gap, providing a method for generating and translating locally relevant evidence of success related to a particular health issue. AI/BCT is a method for turning these “random” successes into “usual” practice at a community level. AI/BCT identifies and speeds up the translation of successful local discoveries into daily practice, reduce suffering, and improve the health of local communities. While our study was limited to implementation of the AI/BCT method across five topics in a single state, we believe our work will help to identify and accelerate the translation of local discovery into regular practice, further reduce unnecessary suffering, and make an overall positive impact on the improved health of local communities touched by this work. To benefit other communities wanting to embark on this work, we have conducted a training in the method and plan to further apply the method with our own community partners.
Footnotes
Author’s Note
John M. Westfall is also affiliated with The Robert Graham Center for Policy Studies in Family Medicine and Primary Care, Washington, DC, USA.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Patient-Centered Outcomes Research Institute (PCORI) Award (ME-1303-5843).
