Sage Journals: Discover world-class research

Abstract

To contribute to healthcare improvements, qualitative health research must adapt to the demanding pace of constantly changing healthcare practices and policies. To meet this challenge, researchers need methods for rigorous and rapid data analysis. This article introduces the Rapid Group Analysis Process (Rap-GAP), a new approach for rapid qualitative data analysis. This method is more efficient than other rapid qualitative analysis methods. It allows for the direct involvement of diverse participants in the analysis process, including patients or healthcare decision-makers with limited qualitative research experience, while keeping the analysis grounded in the primary data (e.g., transcripts). These attributes make Rap-GAP a unique and valuable alternative to traditional qualitative analysis. This article describes the 5-step Rap-GAP process and 3 case studies that demonstrate how to use the method and adapt it for different analytical goals. Future research will evaluate and describe the outcomes of Rap-GAP compared to traditional qualitative analysis.

Keywords

qualitative research qualitative methods rapid qualitative data analysis healthcare health services virtual environments learning health system qualitative evaluation

Introduction

Qualitative research can provide critical insights into both healthcare policy and delivery of care (Chafe, 2017; Hayre & Muller, 2019). However, in the political and economic context of the US healthcare system, decision-making by healthcare leaders is often rapid and sometimes moves forward without clear evidence (Vindrola-Padros & Johnson, 2020). In the ever-evolving healthcare environment, qualitative researchers may feel pressure to share findings more quickly (Vindrola-Padros & Johnson, 2020). This requires researchers to adapt their methods if they want their insights to be integrated into policy and practice change. Though there are concerns about reducing rigor and depth of qualitative work when using more rapid methods, many qualitative researchers acknowledge that increasing the pace at which we are able to conduct our work is important in many contexts and may be essential to ensuring the rich insights provided by our work can impact people’s lives more broadly (Horwood et al., 2022; Neal et al., 2015; Palinkas & Zatzick, 2019).

A recent report from the National Academies of Sciences, Engineering and Medicine on the future of health services research underscores the critical need to focus on rapid-cycle projects “that aim to inform policy makers and health system leaders about issues they face as soon as is practical” (Whicher et al., 2018, p. 2). To be relevant to healthcare leaders, qualitative data must be analyzed and synthesized more quickly than traditional methods allow. In particular, methods for rapid, flexible analysis of qualitative data are needed, as this is often the most time- and resource-intensive phase of qualitative projects.

Traditional, in-depth, qualitative analysis methods produce rich results but require substantial human resources and a generous timeline. Coding qualitative data generally involves one or more trained individuals reviewing the data multiple times — during data collection and code development, throughout coding, and after coding to develop a coding memo — as well as iteratively discussing codes and results with team members to ensure the representation of multiple perspectives and the appropriate breadth and depth for the data and project (Crabtree & Miller, 2002; Creswell, 1998; King, 2012; Saldaña, 2016). Although thorough, the process can take months and sometimes years, delaying dissemination of findings and potentially reducing their relevance for fast-changing healthcare settings.

Recognition that the field needs rapid qualitative analysis techniques is growing. A review by Vindrola-Padros and Johnson identified 15 articles published between 1992 and 2018 that included an adaptation to data collection or analysis that the article authors self-identified as rapid (Vindrola-Padros & Johnson, 2020). The review found that “all articles prioritized either eliminating or speeding up the process of transcribing data to adapt more conventional data collection and analysis methods for a rapid time frame” (Vindrola-Padros & Johnson, 2020, p. 1598).

Rapid ethnography techniques that eliminate transcripts, such as Rapid Assessment Procedure-Informed Clinical Ethnography, and Rapid Qualitative Inquiry (Beebe, 2014; Palinkas & Zatzick, 2019), tend to rely on researchers to summarize data rather than directly analyzing the words and ideas of the individuals from whom the data were collected. Relying on summaries of interpreted data rather than working with direct transcripts in the analysis stage may increase researcher bias (Burgess-Allen & Owen-Smith, 2010). More recent innovations, such as the rapid deductive Consolidated Framework for Implementation Research approach (Gale et al., 2019; Nevedal et al., 2021), and Rapid Identification of Themes from Audio Recordings (RITA) (Neal et al., 2015; Rolf et al., 2021) rely on summarizing and/or developing a priori codes and using notes and an audio recording to fill out matrixes that substantiate themes or constructs. These approaches still rely heavily on either the model constructs or the priorities of individual interviewers to guide what is and is not included.

RITA authors are clear that the method must be very focused and that it does not replace traditional analysis for answering the project’s broader research questions. The authors state, “Although we were also interested in many additional questions for our larger study and although our qualitative data could be used to gain insight into many of these other questions, we maintained a narrow focus on these two foci for our rapid identification of themes” (Neal et al., 2015). Also, RITA focuses on producing code frequencies rather than written quotes or summaries. In summary, these existing approaches to rapid analysis provide valuable tools, but still leave space for other innovations.

With the rising availability of rapid turnaround transcription services and advances in voice recognition software, timely access to high-quality transcripts is becoming less of a barrier for qualitative researchers. Therefore, we believe that the focus on eliminating transcription may not be the most effective or efficient avenue for innovation and change. Although some argue that transcripts may lack nuance and inflection, they still are one of the best ways of documenting qualitative data in a way that preserves the co-construction of meaning between the interviewer and interviewee and the context and nuance of specific statements (although minus intonation). In addition, transcribed quotations allow for transparency in the interpretation process.

Another limitation of existing rapid techniques is they require experienced researchers with large blocks of focused time for study-specific trainings and analyses. This places a significant burden on the qualitative researchers on the study team and limits the ability for important potential contributors and collaborators such as health system leaders and patients to participate in the analysis. These participants may not have qualitative analysis training, but they have unique insights into the subject matter and/or can influence how the findings will be used. Having individuals with varied perspectives (e.g., patients, clinicians, administrative support personnel, etc.) contribute to the analysis has the potential to disrupt reified constructs and highlight ideas from different and unexpected viewpoints at a much earlier stage of the analysis and interpretation process. This can allow for earlier and more seamless integration of input than techniques like member checking, which occur when the interpretive process is often near completion and rethinking constructs might be difficult for both partners or participants and for researchers. Involving collaborators at this stage also generates buy-in from leaders and the people who will ultimately implement and use the findings in practice. Some innovations in qualitative data analysis aim to increase collaborator involvement in analysis, such as a method that uses sticky notes and group discussion to aid in data analysis (Burgess et al., 2021; Kingdon, 1995), and mind mapping that uses a diagram to represent concepts and tasks around a central idea or word (Burgess-Allen & Owen-Smith, 2010). However, the sticky note process is not rapid (it took Burgess and colleagues 349 days), and mind mapping does not produce the detailed connections between themes and verbatim quotes that are critical to demonstrating validity in qualitative findings.

Comparisons of rapid techniques and traditional qualitative analyses found that although rapid methods may reduce the time to manage and analyze data, interpreting and reporting the data can take as much time or longer than traditional methods, reducing any gains in efficiency and applicability (Taylor et al., 2018). Templated abstraction methods such as those described by Gale et al. (2019) allow for summarizing themes with quotes but rely on the interpretation of only a few researchers, without time for discussion or group processing. In addition, although quicker than traditional coding, the techniques are still labor intensive. Gale et al. (2019) reported that the rapid technique using interview summaries took 129 days versus 195 for traditional coding. Of note, in that study, neither templated transcript abstraction nor subsequent categorization of transcript summaries yielded final results.

Analytic strategies employed by human-centered design practitioners such as affinity mapping to cluster similar ideas or concepts from data are indeed rapid, but may be less rigorous than rapid methods developed in academia (Beyer & Holtzblatt, 1997; IDEO, 2022). Techniques used in industry, while speedy, may lack the processes, documentation, and detail required to meet the level of scientific evidence often required for improving healthcare (Morse et al., 2002).

This article introduces a new qualitative analysis approach called Rapid Group Analysis Process (Rap-GAP), which builds and improves on other rapid analysis processes for qualitative methods (Gale et al., 2019; Taylor et al., 2018). We aimed to maximize the benefits of those techniques while minimizing their limitations. Instead of eliminating transcripts, Rap-GAP reduces the time needed to develop a coding list, achieve agreement among coders, and work through a body of qualitative data (often dozens of documents such as transcripts and fieldnotes). Traditionally, one or two people complete this stage of analysis with limited input and accountability. Rap-GAP expands the pool of people who engage with the data and introduces a structured time frame that spreads the work burden, increases accountability, and includes diverse perspectives in data analysis and interpretation. Furthermore, Rap-GAP addresses the recognized need for more guidance on how to perform qualitative analysis virtually (Roberts et al., 2021; Singh et al., 2022). Rap-GAP was developed as a virtual method, leveraging online technology to enhance the quality and speed of the analytic process.

This article contains a detailed, step-by-step description of the Rap-GAP process, explains how it was developed, and summarizes benefits, limitations, and use cases of the method. We provide case studies to illustrate how the method has been used at different phases of the analysis process. We aim to provide a practical guide that allows readers to implement this method in their own research and evaluation settings.

Methods

Rap-GAP was first piloted in May 2020 at the outset of the COVID-19 pandemic. While Rap-GAP was not originally intended to be conducted in a virtual environment, we found that virtual settings and tools increased efficiency by streamlining documentation throughout the process. Specific tools and processes for virtual facilitation are described, and templates are included in supplemental materials or at https://www.act-center.org/download_file/673/0.

Overview

The primary use of Rap-GAP is as a standalone qualitative analysis method for rapid coding, interpretation, and summary of results for dissemination. Rap-GAP analyzes primary qualitative data such as transcripts, observation notes, photos or other data that come directly from research participants or other sources of qualitative data such as field observations. Rap-GAP can also be used to expedite code development as part of a more traditional qualitative analysis process. We note that Rap-GAP was developed in the context of a formal qualitative research or evaluation project and assumes that one or more experienced qualitative researchers are involved in the design, planning, data collection and analysis process. This is not a method in which all participants can be naïve to qualitative data collection and analysis. Also, Rap-GAP was developed in a context in which the preferred coding approach uses and balances both a priori and emergent constructs in the analysis process. It could be adapted for a strictly inductive or deductive approach but our description here provides a general a priori structure while allowing for new and unexpected nuance or constructs to emerge from the data.

Though the Rap-GAP method allows for flexibility and adaptation, the core process involves five steps and requires one or more “leads” to provide structure and accountability to the process. First, the Rap-GAP lead(s) plan for Rap-GAP, select analysis participants, and prepare them to contribute. Second, participants engage with the data individually by reviewing primary data using a series of high-level domains and/or questions to structure reading and identifying insights or key notes of interest. Third, in a group analysis process, participants share their notes and observations, group similar notes into clusters that becomes themes, and discuss questions or conflicting ideas about interpretations or theme identification. Fourth, the Rap-GAP lead(s) collate the documentation from the individual and group processes. Fifth, the Rap-GAP lead(s) write up the process findings. Table 1 and the remainder of this section describe these steps in detail, including guiding questions and key considerations for planning, potential variations in the process, and the roles and responsibilities of lead(s) and participants.

Table 1.

Guiding Questions and Key Considerations for Rap-GAP lead(s) to Plan and Conduct the Rap-GAP Method.

Step	Guiding Questions for Rap-GAP lead(s)	Additional Considerations and Tips
1 Plan and prepare	• Who is our audience and how will we disseminate our findings?	• Rap-GAP participants can be any combination of researchers, project participants, leaders, patients, or other stakeholders but need to have permission to access and view data (may involve IRB, DUAs and/or confidentiality agreements
	• Who should be involved in the Rap-GAP process (e.g., whose perspectives would enhance data interpretation and applicability of findings)?	• Participants do not require research training but may require training on the specific technology or topic
	• What information and training do participants need to fully participate?	• Give participants clear key questions, issues, or conceptual domains to focus on when working with the data
	• What are the key questions or topics that participants should focus on?
2 Engage with data individually	• How should participants interact with the data?	• Ways to engage with data at this step can include reading notes, identifying high-level themes, coding transcripts using a pre-determined code list, or more open-ended identification of notes/insights
	• What kind of themes, quotes or other items should participants pull from the data?
	• What should be documented and where?
	What are participants' time and capacity to engage in the process?
3 Engage with data as a group	• How and how often should the group meet?	• Collaborative analysis brings participants together to share notes, insights, and questions, add comments and notes to a virtual whiteboard, group notes into themes, and talk through ideas
	• Which technology platform(s) will we use to meet and engage with the data?	• At the time of writing, Miro, MURAL, and Jamboards are good options for virtual engagement
	• How should the group process be documented?	• Creating standard processes to label and organize ideas during collaborative analysis will expedite the summarization phase
4 Collate learnings	• How should the data, notes, and themes from steps 2 and 3 be combined and organized?	• Quotes and associated themes should be organized so they are easy to summarize. We recommend assembling results in sortable, searchable form (e.g., an Excel spreadsheet)
5 Summarize findings	• What are the findings of the analysis?	• This step will closely resemble summarization and write up in a traditional qualitative analysis process (e.g., producing a coding memo or qualitative report)
	• What format will be used for the writeup?
	• Who will participate in writing coding memos or reports?

Step 1: Plan and Prepare

Successful Rap-GAP requires thoughtful planning from one or more Rap-GAP lead(s). Lead(s) can be qualitative researchers, project managers, research assistants, or other highly engaged team members. Key qualifications for being a lead include: 1) access to and oversight of the data including tracking whether the data is ready for distribution to participants; 2) organizational skills for scheduling meetings and overseeing timelines and needed communication; 3) excellent communications skills to support training, problem solving and participation in the Rap-GAP process; and 4) a position that enables holding participants accountable for timely completion of tasks. The lead(s) do not need to have qualitative expertise, but it helps if they have participated in a Rap-GAP or other qualitative analysis prior to leading a Rap-GAP so they understand the process from start to finish. The leads are not necessarily the facilitators of the group discussion but need to ensure there are sufficient skilled facilitators to support larger group and breakout discussions. To make the process seamless and efficient, Rap-GAP lead(s) must think through each step and prepare to facilitate the group analysis process. The lead(s) are also responsible for documenting the Rap-GAP process as required by funders and/or for internal reference and evidence of rigor. Table 2 highlights the roles of Rap-GAP lead(s) and participants.

Table 2.

Roles and Responsibilities in the Rap-GAP Process.

	Step 1: Plan and Prepare	Step 2: Engage With Data Individually	Step 3: Engage With Data as a Group	Step 4: Collate Learnings	Step 5: Summarize Findings
Rap-GAP Lead(s): Often the lead qualitative researcher and a project manager or research assistant	• Identify Rap-GAP participants	• Provide instructions and materials (e.g., transcripts, data extraction forms, note templates)	• Prepare a shared virtual space for group engagement	• Assemble and organize individual and group-process results in an organized manner	• Summarize findings from the Rap-GAP process
	• Decide how participants will engage with the data		• Facilitate collaborative analysis
	• Develop training materials		• Facilitate collaborative analysis
Participants: Any project team member and other stakeholders such as clinicians, patients, and organizational leaders	• Attend trainings on the collaborative analysis platform	• Work with the data	• Contribute to the collaborative analysis process	• Share documentation from Step 2	• Review summary and provide feedback to maintain analytic integrity
	• Complete any prep work	• Note key insights and learnings	• Provide clarifying or nuanced context from the data	• Respond to clarifying questions from Rap-GAP lead(s)

The Rap-GAP lead(s) decide who should participate in Rap-GAP and invite them. A unique aspect of Rap-GAP is the inclusion of individuals who may not have formal qualitative analysis training but have valuable perspectives. Involving clinicians, patients, or organizational leaders can enrich the findings and ensure the research is relevant to real-world contexts and applications. Individuals to consider inviting who usually are not involved in traditional coding efforts include: co-investigators focused on quantitative aspects of a project, individuals involved in the design or implementation of the intervention, health system leaders with an interest in the topic or the outcomes of the data, subject matter experts, and research participants or patients. Factors to consider when deciding who to invite include: 1) whether an individual has or can be given access to the data, 2) whether they have the time and interest to read and then talk about the data in a group setting, and 3) how to balance power dynamics and representation across participants. Inviting 1 or more people with facilitation experience can assist the lead(s) in ensuring the group discussions focus on inclusion and sharing ideas and minimize negative power dynamics.

The number of participants can vary. Larger groups (6 or more) may require additional planning to facilitate a productive group analysis process, but more participants reduce the number of transcripts each must review and increases the diversity of insights and robustness of key findings. Our Rap-GAP analysis teams have ranged from 3-9 participants, with each person reviewing 1-6 transcripts. Before finalizing your team, check Institutional Review Board (IRB) determinations and data use agreements to ensure all participants have permission to view data, especially transcripts, which can be hard to completely de-identify. Also, standard ethical processes need to be followed to ensure informed consent and protection of confidentiality. Depending on who is participating in the analysis team and the context of the analysis, transcripts may need to be cleaned of names and other key identifiers before sending them to the Rap-GAP analysis team to protect participant confidentiality. For example, for a program evaluation that includes data from employees, direct supervisors or other leaders may need to be excluded from the analysis team.

The Rap-GAP lead(s) are also responsible for preparing Rap-GAP participants to engage in the process. When planning to train participants, the Rap-GAP lead(s) should consider their familiarity with qualitative analysis, the data content, and the technology they will use. The lead(s) should provide sufficient instructions and support for participants to successfully engage in analysis. Participants should be given clear key questions, issues, or conceptual domains to focus on when working with the data. Participants may benefit from examples they can refer to while working, access to background information, and training opportunities to become familiar with online platforms and tools. Training often includes at least one 30-min session that illustrates how to annotate transcripts including length of annotations and definitions for overarching domains. Another 30-min training session explains how to use technologies such as Excel for spreadsheets and Miro for online collaborative work. Rap-GAP leads should also engage in reflexivity conversations with participants to help them think through how their own positionality may influence what they attend to in the review process. Leads should also be explicit about the theoretical and/or methodological constructs used for interpretation—whether based on phenomenology, constructivism, grounded theory or some other approach.

Step 2: Engage with Data Individually

Rap-GAP participants engage with the data individually before meeting for the group process. Decisions about this phase of analysis are driven by the scope and specificity of the question(s) to answer for the analysis, and how rapid the process needs to be. The Rap-GAP lead(s) should articulate how participants will share the information they learn from engaging with the data.

The lead(s) assign each participant primary data (e.g., transcripts, notes) to review. The amount of assigned data depends on the amount to be analyzed and the group size. Lead(s) may decide to assign participants a different amount of data depending on the experience and/or time the participant has. Participants analyze the data guided by instructions from the Rap-GAP lead(s) and the goals of their product. For example, participants might be asked to read the data and identify high-level insights or key ideas related to pre-determined domains. Notes on insights are documented in a spreadsheet or document that allows participants to add associated quotes (Figure 1). A similar process could be used to develop a code list, but participants are not given pre-determined themes or domains but asked to surface all relevant insights.

Figure 1.

Example of Excel spreadsheet template used for analysis.

A more detailed process might involve coding of transcripts using an a priori code list. For this scenario, the Rap-GAP lead(s) divide transcripts among Rap-GAP participants and provide the code list, instructions for coding, and a form to capture codes and corresponding quotes (e.g., Microsoft Excel spreadsheet, Word document). Before the group analysis process, the Rap-GAP lead(s) gather individual analyses into a shared space (e.g., an online board with virtual sticky notes with notes and representative quotes) for the group to engage with.

Step 3: Engage with Data as a Group

The group analysis process is the most important phase of Rap-GAP. Participants come together in a shared space and post the notes they generated individually for other participants to see and discuss. We used a whiteboard app (Miro.com) that had helpful tools including the ability to create multiple boards on a larger working board, easily generate virtual sticky notes, tag sticky notes, and export the data from the whiteboards into Excel to sort by tags. Depending on the amount of data, number of participants, timeline, and complexity of the analysis, the group analysis process can occur in a single meeting or over multiple sessions. The Rap-GAP lead(s) are responsible for thoughtfully facilitating the process to promote meaningful participation from each participant (e.g., setting an agenda, timekeeping, providing clear instructions and prompts, making time for everyone to share). Depending on the group size and amount of data, lead(s) may find it beneficial to split participants into groups or “breakout rooms” for some portion of the analysis meeting. If some analysis occurs in smaller groups, the lead(s) should provide clear instructions on how groups should engage with the data, what they will be expected to share with the larger group, and how work from small groups will be reintegrated with the whole. Facilitators should be aware of power dynamics and consider ways to support equitable contributions to the discussion such as doing round-robin responses so everyone has a chance to talk and asking participants questions that tap into their areas of expertise. The process of assigning everyone different transcripts and posting sticky notes prior to the group process also has an equalizing effect as everyone’s perspectives are documented without bias toward experience or status, all sticky notes are treated with equal weight, and each person is an expert on the transcript they reviewed.

Participants are responsible for actively contributing to the process by sharing notes/insights generated in Step 2 using virtual sticky notes, discussing those notes with others, and aggregating similar notes/insights and identifying relationships between groups of notes. This work could involve virtually grouping and labeling notes and drawing connections between them or taking notes on cross-cutting themes and high-level relationships. Throughout this discussion, participants should be encouraged to provide clarifying information or nuanced context about their notes or the content of the data they engaged with to preserve analytic integrity.

Consistent processes for organizing and labeling notes and documenting the content on the virtual whiteboards generated during group analysis are critical to support rapid interpretation and consolidation of themes. We found distinct efficiencies in using digital tools for group analysis. For example, some virtual platforms allow users to easily convert cells in a spreadsheet into individual virtual sticky notes. Some platforms allow grouped notes to be “tagged” or labeled for easy sorting. (See the supplemental materials and the templates and instructions that include detailed information about our process at https://www.act-center.org/application/files/1516/8298/5348/Rap-GAP_Instructions_and_templates.pdf).

Step 4: Review and Collate Learnings

After the group analysis process, the Rap-GAP lead(s) pull findings from the individual and group processes together. The “boards” are documented and translated into a spreadsheet or table for rapid consolidation of duplicate notes. (Some virtual platforms automatically convert board content into spreadsheets with notes in one column and tags (themes) in another for easy sorting). Participants either share their documentation from Step 2 or review and add quotes associated with specific themes to the summary document. The product of this step may resemble a hierarchical code list with corresponding quotes or a list of themes with examples from the data (Figure 2). The purpose of this step is to highlight commonalities and consensus while maximizing the ability of individuals to associate quotes with shared themes.

Figure 2.

Example of template with themes and quotes added by Rap-GAP participant.

Step 5: Summarize Findings

The final step of Rap-GAP is summarizing the documentation from Step 4. Depending on the project goals and stakeholder needs, the product could be a high-level summary of key findings, detailed code memo, formal report, slide presentation, or manuscript for publication.

Results

We have used Rap-GAP in six projects to date. For several projects, we adapted the method for different phases of the analysis process and found it to be highly flexible for improving the efficiency of qualitative data analysis. Our distinct uses of Rap-GAP were: (1) group analysis without a pre-developed code list, (2) group analysis with a pre-developed code list, and (3) code list development. We illustrate these uses with case studies to concretely demonstrate each scenario in which we found Rap-GAP to be a useful method.

Case Study 1: Rap-GAP Without a Pre-developed Code List

The initial application of Rap-GAP was for group coding and interpretation without a pre-developed code list. We applied Rap-GAP to a qualitative evaluation of the Community Resource Specialist (CRS), a new role in primary care that addresses social health needs by connecting patients with community resources (Hsu et al., 2018). The evaluation was part of Kaiser Permanente Washington’s Learning Health System program, a strategic initiative to rapidly improve care and services for patients by integrating research capabilities into strategic decision-making in care delivery (Allen et al., 2021). Data were from 44 patient interviews of approximately 30–45 minutes that were audio-recorded and transcribed. The Rap-GAP leads were a senior qualitative researcher with more than 20 years of qualitative data analysis experience and a more junior team member to support communication, data management and final write-up. The full Rap-GAP team also contained a project manager who was learning qualitative methods and participated in the data collection process, two senior researchers on the project who were familiar with qualitative methods but whose role on the project was not primarily qualitative, a project manager, three other project team members who had experience with qualitative methods but did not participate in the data collection, and an administrative support person who was new to qualitative research but who had a deep knowledge of the project.

Step 1: Plan and Prepare

To enhance the applicability of evaluation results in the context of learning health systems research, the Rap-GAP leads engaged care delivery leaders, learning health system leaders, and qualitative and quantitative evaluation team members. The 9 participants were each given 4-5 transcripts from patient interviews. The leads created, tested, and iterated a system for facilitating participant engagement with the data. They gave each participant an Excel spreadsheet for individual analysis organized by domains of interest to the evaluation. Space for notes from the data and associated quotes was organized by domain (Figure 1). In this case study, domains were pre-determined by the study team and specific notes were generated by participants from their review of the data. Participants were given detailed instructions for reviewing the data and using the spreadsheet.

To prepare for Step 3, group analysis, Rap-GAP leads set up a Miro board with a “frame” (an organizing tool in Miro) for each domain. The Rap-GAP leads also hosted a practice session to provide coaching on using the analysis spreadsheet and Miro.

Step 2: Engage with Data Individually

Rap-GAP participants read their assigned transcripts and added notes and associated quotes to their Excel spreadsheet under the relevant domains (Figure 2).

Before the group meeting in Step 3, individuals copied and pasted notes they identified into the Miro board frame that matched the domain from their spreadsheet. These notes appeared as sticky notes on the board (Figure 3).

Figure 3.

Example of notes/insights as sticky notes within frames on the Miro board.

Step 3: Engage with Data as a Group

Group analysis occurred in one 2-h virtual meeting. Participants worked in small groups to collect similar and related notes within each domain. Small groups shared their results with the larger assembly, which held discussions to reach agreement on groupings (Figure 4).

Figure 4.

Example of grouped themes after group process and tagging.

Step 4: Review and Collate Learnings

Themes grouped during the analysis process were labelled using “tags” in Miro. The final set of group sticky notes with tags were downloaded into a spreadsheet. Miro automatically organized the data by frame/domain and created separate columns for sticky notes (notes), and tags (theme groups). Figure 5 The Rap-GAP leads manually created separate tabs for the domains and added columns for quotes. Participants manually added quotes from their individual spreadsheets to the summation spreadsheet, linking the notes to the source quotes.

Figure 5.

Example of final collated dataset after analysis, download from Miro, and manual organization with quotes.

Step 5: Summarize Findings

Findings were summarized and presented to delivery system leaders in a “slidedoc”(Duarte, 2023), a data placement, and a standard slide presentation (Lysy, 2021; Pankaj, 2016). Table 3 Findings were also integrated into a CRS Playbook (available at https://www.act-center.org/), a document providing guidelines and instructions for implementation of the CRS role (Lee et al., 2022; Ronzio, 2021).

Table 3.

Analysis Timeline-Case Study #1.

Timeframe & step	Description of Task
Day 1 (end of Step 1)	Transcripts assigned (n = 44)
Days 2–8 (Step 2)	Individuals review and analyze data
Day 9 (Step 3)	Group meeting to review/discuss themes
Day 76 (Steps 4 and 5)	Final summary and report

Benefits and Challenges of Rap-GAP in Case Study 1

Using Rap-GAP for analysis circumvented the code development process, which can take several months or longer. We were also able to engage a large group of stakeholders in data analysis and sense-making discussions, so decision-makers understood the data thoroughly and could identify opportunities to integrate findings directly into the health system setting. Finally, the process took significantly less time than traditional coding, with the full process taking only 76 days.

Challenges were related to this being the first application of Rap-GAP. They included learning the appropriate amount of information to include on a sticky note, particularly for those unfamiliar with annotating qualitative data, ensuring consistency of data documentation and labeling across all participants (e.g., consistent labeling of quotes with participant ID number), and maximizing efficiency of the tools in the whiteboard software.

Case Study 2: Rap-GAP With a Pre-developed Code List

In this case study, Rap-GAP was used for a study that explored the experiences of bariatric surgery patients with a focus on racially and ethnically diverse patients. Rap-GAP was used to generate targeted preliminary findings for conference presentations from a subset of the 68 available transcripts. A code list had already been drafted using more traditional code list development processes, including review of transcripts and stakeholder input.

Step 1: Five team members volunteered to participate in the Rap-GAP. The Rap-GAP leads were the qualitative lead for the project who had 10 years’ experience in qualitative research, and a qualitative mentor with more than 20 years of experience. The Rap-GAP team also included a clinician with experience in qualitative data analysis, and a project manager and a research associate, neither of whom had prior experience with qualitative data analysis. To train the Rap-GAP team, the Rap-GAP leads held a 1-h, interactive, online training session. Each participant was assigned 3-4 transcripts from a set of priority patients and was given a set of 12 codes to use from a single domain in the code list about patients’ interactions with the healthcare system and their care teams.

Step 2: Team members reviewed one transcript and then met to discuss questions and ensure a shared understanding of the individual analysis process. After this initial discussion, they coded their remaining assigned transcripts and transferred quotations into an Excel spreadsheet with a column for each code.

Step 3: The Rap-GAP lead set up a Miro board with a frame for each code. Team members transferred quotations into Miro and onto the appropriate frame. The team met for 2 hours to discuss, group, and tag quotes.

Steps 4 & 5: Quotes were exported to Excel and organized with a tab for each code. Table 4 Quotes and subcodes were assigned to each code. This document was used to engage patient stakeholders in discussion and to develop presentations that were shared at a national academic conference.

Table 4.

Analysis Timeline-Case Study #2.

Timeframe & step	Description of Task
Day 1 (end of Step 1)	Transcripts assigned (n = 16)
Day 5 (Step 2)	Review of first transcript and the Rap-GAP process
Day 9 (Step 3)	Group meeting to review/discuss coded quotations
Day 43 (Steps 4 and 5)	Final summary and final presentations completed

Benefits and Challenges of Rap-GAP in Case Study 2

The key benefit of using Rap-GAP in this circumstance was speed. Rap-GAP facilitated a quick coding process with targeted codes, yielding quotes that resulted in rich findings that could be shared at a national conference. It also allowed the team to include individuals who would not have been able to engage in a traditional coding process. A challenge was having long quotes on sticky notes that were hard to read and interpret in the short time allocated for the group discussion.

Case Study 3: Rap-GAP for Code List Development

The final example used Rap-GAP to develop a code list. Code list development can often require many iterations to ensure the list is complete and coders have a shared understanding of the codes. In this case, Rap-GAP was used to speed the code development process before the team conducted traditional coding. The project involved interviews with youth, parents and providers who had participated in an intervention to reduce off-label antipsychotic use.

Step 1. The Rap-GAP process participants were four individuals who were members of the core qualitative analysis team. All but one were experienced qualitative researchers and had conducted at least some of the interviews. Each of the four Rap-GAP participants was asked to review and free code (associate descriptive terms to text without a code list) the same four transcripts.

Step 2. Free coding was recorded in a single column in an Excel spreadsheet and transferred to a Miro board as individual sticky notes.

Step 3. The team met for 2 hours to review, sort, and group the notes (i.e., text on the sticky notes). They discussed the passages that generated their notess if other team members did not understand it. Notes were grouped within each frames and tagged with theme labels.

Step 4. Miro board results were downloaded into Excel and further consolidated and refined. The team did a final round of review, providing input on the resulting code list.

Step 2. & 3: To finalize the code list for individual use, the team moved back to engaging with the data individually. All members of the coding team individually coded several transcripts and then compared their coding.

Step 5. Once all members of the team demonstrated a shared understanding of the codes and of the team’s overall approach to the data, Table 5 the code list was considered final, table and individuals were assigned transcripts to code.

Table 5.

Analysis Timeline-Case Study #3.

Timeframe & step	Description of Task
Day 1 (end of Step 1 & Step 2)	Team assigned three transcripts to free code for code list
Day 16 (Step 3)	Team meets to discuss themes/notes
Day 22 (Step 4)	Draft code list shared and reviewed by team
Day 52 (Step 2 & 3)	Team coded and compared transcripts
Day 73 (Step 5)	Code list finalized and individual coding started

Benefits and Challenges of Rap-GAP in Case Study 3

The process this team previously used to develop code lists involved one person drafting the code list, sharing it with the other coders, and doing successive rounds of coding using the same transcript to refine the code list and code definitions. Using Rap-GAP for code development allowed the team to reach a shared understanding on a code list after far fewer rounds of coding and discussion than their previous process. A challenge was that one participant initially was reticent to fully participate due to lack of confidence. This participant quickly overcame hesitations after experiencing the first group session and fully participated in a subsequent session.

Discussion

This article details Rap-GAP, a flexible method for rapid qualitative data analysis. Rap-GAP draws techniques from a number of rapid analysis processes, combining them in a unique way that provides an overarching structure but allows for significant adaptation and customization. We provide step-by-step instructions for carrying out a Rap-GAP process, provide three case studies that illustrate a variety of specific uses of Rap-GAP, and supply an instruction manual with templates https://www.act-center.org/download_file/673/0. These resources are also in supplemental materials.

These case studies demonstrate the benefits of Rap-GAP including: (1) allowing researchers to engage more people in the sense-making process than is customary, including people who may not have formal qualitative analysis training; (2) providing mutual accountability for showing up to discussions with analysis tasks completed and ready to engage and discuss; (3) allowing participants to engage directly with transcripts and other detailed primary data; and (4) decreasing the time needed for analysis. Indeed, in Case Study 1, the team analyzed 44 patient transcripts and wrote up the results in less than 3 months. These case studies also reflect the flexibility of Rap-GAP, which is fundamentally about engaging all relevant stakeholders in sense-making, having robust discussions about the data and themes used to group them, and summarizing those themes. We also encountered challenges and areas for continued improvement and innovation, including providing brief but sufficient training for all participants, empowering those with less experience or confidence to speak up and assert their perspectives, and giving patient and community partners access to raw data. Two case studies experienced challenges with how much information to include on a single sticky note. As we have continued to practice and refine Rap-GAP, we found that for maximum clarity, notes/sticky note text should be approximately 5–15 words, have a clear positive or negative valence (if appropriate), and provide a sense of context. For example, “experience referring to mental health” does not provide a valence or sense of whose perspective this represents, whereas “participant had negative experiences referring to mental health” adds only 3 words but tells whose perspective is represented and whether it was a positive or negative experience.

Rap-GAP addresses unique challenges for rapid analysis. First, it uses primary data, including transcripts, which are increasingly easy to get rapidly, without delaying the analysis process (Vindrola-Padros & Johnson, 2020). Not using the primary data can result in the loss of participant voices and add unseen layers of interpretation that can be detrimental to analytic validity. Second, by involving more people, Rap-GAP distributes work and creates mutual accountability. This distribution of work is not usually seen in other rapid analysis processes that frequently focus on a small team of researchers tasked with time-consuming, detailed analytic tasks to complete under tight timelines. Rap-GAP also facilitates conversation and discussion among participants that can enrich the analysis. Finally, Rap-GAP significantly reduces the time between completion of data collection and the final summary or write-up of findings. In contrast, some rapid processes can take many months to reach final write-up (Figure 6).

Figure 6.

Rap-GAP timeline compared to analysis timelines in Gale et al. (2019).

We also found that Rap-GAP was affordable since it did not demand excess time commitments from study team members and spread the time needed across more people. (We found it is often easier for 10 people to dedicate 5 hours each than for 1 person to dedicate 50 hours to analysis.) Although including diverse collaborators can mean inviting leaders with higher salaries, the manageable time commitments meant that we did not see pronounced budget implications. The process can also circumvent the use of expensive qualitative data software.

Rap-GAP is useful for research projects that would benefit from broad participation and/or rapid analysis. It is particularly useful for evaluation and formative research, which often require quick turnaround of findings. Rap-GAP may not be the best fit for research that requires rich detail and attention to nuance. For example, if the analysis goal is exploring the nuanced meanings of a single concept or experience, such as the experiences of bariatric patients in sharing food and social eating, Rap-GAP may not surface themes in enough detail for full exploration. Nonetheless, in our experience, Rap-GAP allows inclusion of qualitative findings into rapidly moving evaluation, design and health systems change in a way that would be impossible to integrate using traditional analysis timelines. Because Rap-GAP is accessible, timely, and can be broadly inclusive, it is well suited to advance the rapid integration and application of qualitative research into healthcare practice and health system organizational decision-making.

Also of note, Rap-GAP was conceptualized before the COVID-19 pandemic, and we intended to conduct it in person using paper sticky notes, flip charts, and other hands-on idea-generation tools. However, when Rap-GAP was piloted in May 2020, we had to adapt it for a virtual environment. Serendipitously, we found the virtual tools streamlined documentation throughout the process (e.g., saving ideas on virtual sticky notes, efficiently collating notes and quotes using spreadsheets). Similar to other virtual qualitative analysis experiences, we found virtual platforms and tools useful as long as participants had clear instructions for using them (Roberts et al., 2021; Singh et al., 2022). Although Rap-GAP was not designed to be exclusively virtual, to date it has been done only virtually. We believe Rap-GAP would function well as an in-person method and some of the technical tools we describe could still be used in person, to keep the method as rapid as possible.

The Rap-GAP process also has limitations. We continue to experiment with how to integrate reflexivity and acknowledge power dynamics among the Rap-GAP participants and between the group and other power brokers and meaning makers. Any interpretation exercise has inherent biases, and being more explicit and conscious of these is an area we are continuing to experiment with and explore. Also, we encountered challenges giving access to transcript data (even when cleaned) to individuals who are not employees of our research institute or are not involved in projects covered by an existing data use agreement. Sharing data, even for quality improvement work, may be difficult because IRBs and organizational confidentiality agreements may restrict giving raw data to individuals external to the organization in which the data was collected. This limitation can be addressed in a variety of ways including data use and confidentiality agreements, however these strategies can require considerable administrative effort and need to be planned for well in advance. A partial solution we found to this challenge is to have participants who do not have access to the data attend the group sessions and help group the sticky notes into themes. Finally, we have not yet formally conducted a direct comparison of Rap-GAP to traditional qualitative analysis. However, comparisons of other rapid analysis processes that share theoretical underpinnings with Rap-GAP found that while they might not surface as many themes, most were identified, with significant overlap between the themes produced through rapid and traditional processes (Taylor et al., 2018). Studies of rapid methods also found they can achieve analytical depth and rigor (Gale et al., 2019; Taylor et al., 2018; Watkins, 2017).

Conclusion

Rap-GAP is an easy-to-use, stepwise process that both engages a wide array of team members and participants in the qualitative analysis process and shortens the timeline for qualitative analysis compared to traditional methods. We are planning studies that compare Rap-GAP to traditional qualitative analysis. Our findings may pave the way for broader use and acceptance of Rap-GAP, especially in peer-reviewed publications of evaluation and quality improvement studies. Rap-GAP is useful for raising the efficiency of qualitative analysis while also increasing the quality and depth of results by including input from multiple perspectives, and use of primary data and quotations in findings. We encourage other health services researchers to use this method to promote integration of qualitative data into clinical quality improvement, evaluation, and decision-making in rapidly evolving healthcare settings.

Supplemental Material

Supplemental Material - Rapid Group Analysis Process (Rap-GAP): A Novel Approach to Expedite Qualitative Health Research Data Analysis

Supplemental Material for Rapid Group Analysis Process (Rap-GAP): A Novel Approach to Expedite Qualitative Health Research Data Analysis by Clarissa Hsu, Jessica Mogk, Laurel Hansell, Joe Glass, and Claire Allen in International Journal of Qualitative Methods

Footnotes

Acknowledgements

Thank you to all the teams and participants in Rap-GAPs who helped further refine this process. We thank Drs. Karen Coleman, Cara Lewis, and Robert Penfold for allowing the use of these innovative methods on their projects and permitting us to highlight this work in our case studies. We would also like to thank the Kaiser Permanente Washington Learning Heath System and the Center for Accelerating Care Transformation at Kaiser Permanente Washington Health Research Institute for supporting innovation in health services research. Lorella Palazzo provided detailed review of a draft manuscript, and members of the Kaiser Permanente Washington Health Research Institute’s Qualitative Research Program provided feedback on the methods section. Finally, we thank Chris Tachibana for her skilled scientific editing and James Licitra and David Hsu for assistance with the references and formatting of this paper.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the We want to recognize Kaiser Permanente Washington for supporting time to work on this manuscript. We would also like to acknowledge the following funding that supported the work highlighted in two of the case studies in this manuscript: National Institute on Minority Health and Health Disparities, 1R01MD013874, and National Institute of Mental Health, HHSN271201600004C. Kaiser Permanente Washington’s Learning Health System provided the funding for the evaluation featured in the remaining case study.

Ethical Statement

ORCID iDs

Clarissa Hsu

Jessica Mogk

Supplemental Material

Supplemental material for this article is available online

References

Allen

Coleman

Mettert

Lewis

Westbrook

Lozano

(2021). A roadmap to operationalize and evaluate impact in a learning health system. Learning Health Systems, 5(4), Article e10258. https://doi.org/10.1002/lrh2.10258

Beebe

(2014). Rapid qualitative inquiry. A field guide to team-based assessment (2nd ed.). Rowman and Littlefield.

Beyer

Holtzblatt

(1997). Contextual design: Defining Customer-centered systems. Morgan Kaufmann Publishers Inc.

Burgess

Jongbloed

Vorobyova

Grieve

Lyndon

Wesseling

Salters

Hogg

R. S.

Parashar

Pearce

M. E.

(2021). The “sticky notes” method: Adapting interpretive description methodology for team-based qualitative analysis in community-based participatory research. Qualitative Health Research, 31(7), 1335–1344. https://doi.org/10.1177/10497323211002489

Burgess-Allen

Owen-Smith

(2010). Using mind mapping techniques for rapid qualitative data analysis in public participation processes. Health Expectations: An International Journal of Public Participation in Health Care and Health Policy, 13(4), 406–415. https://doi.org/10.1111/j.1369-7625.2010.00594.x

Chafe

(2017). The value of qualitative description in health services and policy research. Healthcare Policy, 12(3), 12–18. (Valeur de la description qualitative dans la recherche sur les politiques et services de santé.

Crabtree

Miller

(Eds.), (2002). Doing qualitative research (2nd ed.). Sage Publications, Inc. https://www.qualitative-research.net/index.php/fqs

Creswell

J. W.

(1998). Qualitative inquiry and research design: Choosing among five traditions. Sage Publications.

Duarte

(2023). Slidedocs. https://www.duarte.com/resources/books/slidedocs

10.

Gale

R. C.

Erhardt

Bounthavong

Reardon

C. M.

Damschroder

L. J.

Midboe

A. M.

(2019). Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementation Science: IS, 14(1), 11. https://doi.org/10.1186/s13012-019-0853-y

11.

Hayre

Muller

(2019). Enhancing healthcare and rehabilitation: The impact of qualitative research. CRC Press.

12.

Horwood

Luthuli

Chiliza

Mapumulo

Haskins

(2022). It’s not the destination it’s the journey: Lessons from a longitudinal ‘mixed’ mixed-methods study among female informal workers in South Africa. International Journal of Qualitative Methods, 21, Article 160940692211237. https://doi.org/10.1177/16094069221123718

13.

Hsu

Hertel

Johnson

Cahill

Lozano

Ross

T. R.

Ehrlich

Coleman

BlueSpruce

Cheadle

Matthys

Chapdelaine

Gray

Tufte

Robbins

(2018). Evaluation of the learning to integrate neighborhoods and clinical care project: Findings from implementing a new lay role into primary care teams to address social determinants of health. The Permanente Journal, 22, 18-101. https://doi.org/10.7812/tpp/18-101

14.

IDEO . (2022). IDEO design kit. https://www.designkit.org/methods/find-themes

15.

King

(2012). Doing template analysis. In Symon

Cassel

(Eds.), Qualitative organizational research: Core methods and current challenges. Sage Publications.

16.

Kingdon

J. W.

(1995). Agendas, alternatives, and public policies/John W. Kingdon. Longman.

17.

Lee

Mogk

DeWitt

Parish

Lewis

Hsu

(2022). Community resource specialist integration Playbook. https://www.act-center.org/application/files/8716/4859/5567/CRS_Playbook_Public.pdf

18.

Lysy

(2021). What is a Slidedoc? https://freshspectrum.com/slidedocs/

19.

Morse

Barrett

Mayan

Olson

Spiers

(2002). Verification strategies for establishing reliability and validity in qualitative research. International Journal of Qualitative Methods, 1(2), 13–22. https://doi.org/10.1177/160940690200100202

20.

Neal

J. W.

Neal

Z. P.

VanDyke

Kornbluh

(2015). Expediting the analysis of qualitative data in evaluation: A procedure for the rapid identification of themes from audio recordings (RITA). American Journal of Evaluation, 36(1), 118–132. https://doi.org/10.1177/1098214014536601

21.

Nevedal

A. L.

Reardon

C. M.

Opra Widerquist

M. A.

Jackson

G. L.

Cutrona

S. L.

White

B. S.

Damschroder

L. J.

(2021). Rapid versus traditional qualitative analysis using the consolidated Framework for implementation research (CFIR). Implementation Science: IS, 16(1), 67. https://doi.org/10.1186/s13012-021-01111-5

22.

Palinkas

L. A.

Zatzick

(2019). Rapid assessment procedure informed clinical ethnography (RAPICE) in pragmatic clinical trials of mental health services implementation: Methods and applied case study. Administration and policy in mental health, 46(2), 255–270. https://doi.org/10.1007/s10488-018-0909-3

23.

Pankaj

Emery

A. K.

(2016). Data placemats: A facilitative technique designed to enhance stakeholder understanding of data. In Evaluation and facilitation (pp. 81–93). New Directions for Evaluation.

24.

Roberts

J. K.

Pavlakis

A. E.

Richards

M. P.

(2021). It’s more complicated than it seems: Virtual qualitative research in the COVID-19 era. International Journal of Qualitative Methods, 20, Article 160940692110029. https://doi.org/10.1177/16094069211002959

25.

Rolf

Campbell

Thompson

Argus

(2021). Australians’ experience of the COVID-19 pandemic: Advantages and challenges of scaling up qualitative research using large-scale rapid analysis and building research capacity across rural Australia. International Journal of Qualitative Methods, 20, Article 16094069211051937. https://doi.org/10.1177/16094069211051937

26.

Ronzio

(2021). What's a business Playbook? [How to write one in 4 easy steps] . https://trainual.com/manual/what-is-a-business-playbook-and-how-to-use-it-to-scale#:∼:text=A_business_playbook_is_a,%2C_business_strategy%2C_and_differentiator

27.

Saldaña

(2016). The coding manual for qualitative researchers (3rd ed.). Sage Publications.

28.

Singh

Tang

Thombs

Armas

Nie

J. X.

Nelson

M. L. A.

Gray

C. S.

(2022). Methodological insights from a virtual, team-based rapid qualitative method applied to a study of providers' perspectives of the COVID-19 pandemic impact on hospital-to-home transitions. International Journal of Qualitative Methods, 21, Article 16094069221107144. https://doi.org/10.1177/16094069221107144

29.

Taylor

Henshall

Kenyon

Litchfield

Greenfield

(2018). Can rapid approaches to qualitative analysis deliver timely, valid findings to clinical leaders? A mixed methods study comparing rapid and thematic analysis. BMJ Open, 8(10), Article e019993. https://doi.org/10.1136/bmjopen-2017-019993

30.

Vindrola-Padros

Johnson

G. A.

(2020). Rapid techniques in qualitative research: A critical review of the literature. Qualitative Health Research, 30(10), 1596–1604. https://doi.org/10.1177/1049732320921835

31.

Watkins

D. C.

(2017). Rapid and rigorous qualitative data analysis: The “RADaR” technique for applied research. International Journal of Qualitative Methods, 16(1), Article 160940691771213. https://doi.org/10.1177/1609406917712131

32.

Whicher

Rosengren

Siddiqi

Simpson

(2018). The future of health services research: Advancing health systems research and practice in the United States (p. 2). National Academies Press (US).

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.56 MB