Abstract
Involving communities in research prioritization is an important component of developing relevant research, policy, and healthcare questions. The coronavirus disease 2019 (COVID-19) pandemic has led to an increased need for research prioritization methods which align with public health protections. The Research Prioritization by Affected Communities (RPAC) protocol is a participatory method which directly involves historically excluded communities in developing and prioritizing research questions. We adapted the RPAC protocol for the virtual environment (RPAC-ve) to understand the research, healthcare delivery, and policy priorities of greatest importance to communities affected by the COVID-19 pandemic in relation to pregnancy in the United States. RPAC-ve included the use of a web-based platform, sharable documents for listing and ranking research questions, and a scribe to capture a visual representation of RPAC-ve sessions. RPAC-ve is a flexible and effective priority setting method which can be used to engage high risk populations.
Background
In 2019, Severe Acute Respiratory Syndrome Corona-Virus (SARS-Co-V-2), also known as COVID-19, emerged as a novel viral illness, and on March 11, 2020, the World Health Organization (WHO) declared a global pandemic. The COVID-19 pandemic has shuttered economies worldwide and exposed long-standing inequities that are associated with structural racism and divestment from crucial public health infrastructure (Poteat et al., 2020; Tan et al., 2022). During the pandemic, existing inequities continued to be highlighted in Black, Indigenous, People of Color (BIPOC) and queer-identified communities in the United States (U.S.). However, BIPOC and queer communities were systematically excluded from early COVID-19 public health relief efforts (Emerson & Montoya, 2021; Siegel et al., 2022) leading to significant inequities in COVID-19 morbidity and mortality (Bassett et al., 2020; Selden & Berdahl, 2020).
Community involvement in research, healthcare, and policy priority setting is crucial for ensuring equitable public health response for the COVID-19 pandemic and future public health crises. Various strategies have been used for priority setting (Iqbal et al., 2021), but few have been developed specifically for communities impacted by health inequities. In this study we used a modified version of the Research Priorities of Affected Communities (RPAC) method (Franck et al., 2018). The RPAC protocol draws on principles of the James Lind Alliance (JLA) approach (Reay et al., 2014) which uses a multiphase process to engage patients, caregivers, and clinicians in identifying treatment uncertainties (James Lind Alliance, 2021). In brief, the JLA protocol involves; (1) developing a prior setting partnership and steering committee with relevant stakeholders; (2) initial information gathering from stakeholders via survey methods and reviewing existing literature (e.g., systematic reviews, clinical guidelines); (3) processing and categorizing list of submitted uncertainties; (4) interim priority setting to reduce list of uncertainties; and (5) in person meetings with stakeholder groups to arrive at a final list of 10 research priorities. Recognizing systemic power inequities in the JLA approach, the RPAC protocol was first developed by Franck and colleagues to center voices of women of color disproportionately impacted by adverse maternal and infant health outcomes (Franck et al., 2018, 2020). RPAC differs from the JLA approach in that is solely focuses on the communities most affected by the health condition, questions are primarily generated within group sessions, and the protocol is designed to understand more deeply about the experiences and contexts of those most affected by health inequities.
Due to the COVID-19 pandemic, we modified the original in-person RPAC protocol (Franck et al., 2018) to accommodate public health protections. We developed the RPAC virtual environment (RPAC-ve) and held sessions with community stakeholders via Zoom from November 2020 to January 2022. The RPAC-ve protocol allowed us to gain a deeper understanding of priorities important to BIPOC and queer individuals with the capacity for pregnancy at the height of multiple COVID-19 surges in the U.S. Public health responses were inattentive to the unique measures necessary to protect this population from the virus while also maintaining their mental, physical and social well-being (McCray & Rosenberg, 2021; Purtle, 2020). The purpose of this paper is to describe the RPAC-ve protocol used to assess these urgent, unmet needs.
Methods
We developed and implemented RPAC-ve as an extension of PRIORITY (Pregnancy CoRonavIrus Outcomes RegIsTrY), a national study launched in March 2020 to document the impacts of COVID-19 on pregnant people and their newborns (Afshar et al., 2020; Flaherman et al., 2020; Jacoby et al., 2021). RPAC-ve was implemented by members of the Reproductive Health Equity and Birth Justice Core (RHE&BJ). The purpose of the RHE&BJ Core was to link essential on-the-ground, regional infrastructure to national initiatives that should have lasting implications for innovations in public health. The RHE&BJ Core supported the community-based organizations (CBOs) via technical and financial assistance to analyze gathered data to develop site-specific interventions that can be tested and evaluated specific to health promotion, participation in public health surveillance activities for COVID-19, and targeted messaging that should help rebuild community trust in public health systems.
Description of CBOs Included in RPAC-ve Implementation.
Protocol
Comparison of RPAC Versus RPAC-ve.
RPAC-ve Session One Protocol.
Session One
Preparation
We prepared for session assigning roles, jointly developing a recruitment strategy, scheduling sessions, and confirming remuneration with CBO staff. We also met with each CBO to adapt the RPAC-ve protocol. During this phase the CBO liaison was responsible for recruiting and confirming participants. The RHE&BJ Core provided CBOs with social media tool kit with sample recruitment posts for use on Facebook, Twitter, and Instagram. CBOs were also given a recruitment script for communicating with members of the communities they serve via phone or email. The RPAC-ve protocol differed from the original protocol in that there was no arrangement of a physical meeting space, childcare, and meals. To prepare for virtual meetings, we scheduled session times with CBO partners, and sent Zoom invitations to facilitators, CBO partners, recruited participants.
Logistics
Session One logistics involved introductions by the RHE&BJ Core, CBO staff, and participants, as well as grounding, review of the RPAC-ve protocol, and explaining next steps and close out. During introductions RHE&BJ team members shared their names, pronouns, a bit about themselves, and their assigned role for the session. Participants were also encouraged to introduce themselves. Introductions were followed by a grounding exercise and review of ground rules. During this time participants had the opportunity to ask questions and/or express concerns about the RPAC-ve protocol. The grounding exercises were led by the RHE&BJ Core and served to refocus all group attention to the present moment in preparation for engaging in discussions on participant’s COVID-19 experiences.
The RPAC-ve protocol included the use of a visual scribe, whose role was to capture a visual representation of participant’s narratives during the research question-generation process. The RHE&BJ Core hired an experienced artist/illustrator who was provided with an overview of the RPAC-ve protocol. The visual scribe attended sessions with all CBOs and created visual summaries following each session. The visual renderings included words and images to summarize and draw connections between participant research questions (Figure 1). Approximately an hour of Session One was allocated to facilitated research question generation. A RHE&BJ Core member facilitated this process. To close out the meeting, a RHE&BJ facilitator reviewed the list of generated questions with participants for accuracy and invited participants to give closing remarks. An image of some of the research questions developed by participants in an RPAC-ve session. The image was developed by visual scribe Ashanti Gardner.
Session Guide
The lead RHE&BJ facilitator began sessions by providing a description of the research question development process. This helped to encourage dialogue with the RHE&BJ team. We prepared examples that focused on exploring pregnancy and childbirth during the COVID-19 pandemic, views about healthcare, research, and policies specific to registries, and uncertainties about COVID-19 to orient participants to the process. Unique to RPAC-ve, participants had the opportunity to engage in discussion via the chat feature or by using emojis (e.g., hearts to reflect empathy or support) in the virtual Zoom environment. These features allowed for both nonverbal participation and the RHE&BJ team to determine if certain moments resonated with the group more than others based on chat interactions or the use of emojis. Before introductions, participants were given instructions on how to use the chat, raise your hand, and muting functions.
Next, participants were asked to introduce themselves and to tell their family and birth story in whatever way felt meaningful to them. The facilitator asked probing questions (e.g., Can you tell me more about that?) that invited participants to share more details about their experiences. The facilitator reflected participants experiences back to them in the form of research questions. This process helped uncover participants underlying questions and uncertainties. The facilitator guided participants to turn their statements into potentially researchable questions. For example, many participants discussed confusion about isolation and quarantine and the facilitators would state those concerns back to participants as a question: Participant: Well, it’s like okay, when you’re diagnosed [with COVID-19] and you’re just told to quarantine. “What do I do while I’m quarantined?” You didn’t give me anything. You didn’t say, “Maybe take,” because ultimately, I ended up treating it as if it was a common cold by taking cold medication, nighttime meds, vitamins. Coming up from the country we do home remedies. So, we come up with things like hot toddies or, you know, drink some liquor, get under a bunch of covers, sweat it out. We don’t know what to do exactly, and it kind of leaves you, --well me--, I’ll speak for me. It kind of left me feeling like almost helpless. I was confused. I was just a bunch of emotions and a bunch of things going through my mind like, I don’t know if this will be enough to handle the situation that I’m going through. And thank God for friends and family who are supportive and who would drop things off like orange juice or fruit. Because for me, I didn’t have a huge appetite, but I know I needed to eat, naturally. They would drop off -- go ahead, I’m sorry. Facilitator: No, no, no. It sounds like there’s also a ton of questions around symptoms, right? So, “What is the most effective way to manage symptoms when you’re in quarantine?”
As conversations progressed members of the RHE&BJ team documented generated questions and the visual scribe observed the interactions amongst participants. At the end of the first session, a preliminary list of questions was developed. The process typically began slowly, however, once participants observed the first person, they quickly began to understand the process and start dropping questions in the chat or talking amongst each other.
Interim Analysis
In between Sessions One and Two, the RHE&BJ team reviewed the audio recordings and transcripts to identify additional questions not captured by the research team during the initial session. RHE&BJ team members removed any duplicate questions and combined similar questions. The list of questions was then separated into two lists, “research questions with answers” and “researchable questions”. Research questions with answers are those for which there are definitive answers from systematic reviews or professional guidelines or regulations. Thematic analysis (Braun & Clarke, 2006) was then used to organize the list of “researchable questions” to identify themes. Themes were used to categorize the list of questions.
Session Two
Preparation
Session Two Protocol.
Note: (a) The protocol steps for sessions 1 and 2 can be repeated within and across geographic regions. If this is done, then additional thematic analysis can be done to merge the question list and to explore similarities or differences in priority topics and questions across groups with similar or differing sociodemographic characteristics; (b) Over the course of the sessions, rich qualitative data about the participants’ health and healthcare experiences will be generated in the discussion. These data are recorded and transcribed along with the research questions and topics. These data may be subjected to qualitative analysis, using thematic analysis or another framework.
Logistics
Session Two began with greetings and introductions by the RHE&BJ team. Then the RHE&BJ facilitator shared the full list of questions generated at the first session and explained the process that was used to prioritize the list of questions generated in the previous session. During this time, participants also had the opportunity to ask questions about the process.
Session Guide
The RHE&BJ team members read to participants the full list of generated questions grouped by themes. Participants had the opportunity to review and revise the list of generated questions and themes prior to engaging in research question prioritization. For the first round of prioritization, participants were given 10 minutes to identify their top ten questions on the Google Docs file or in the chat feature of the Zoom platform.
After the first round of prioritization, topic rankings were grouped, and lower priority questions were removed from the working Google Docs file. The full lists of questions across groups were maintained in separate Google Docs files, and participants were reminded that all questions are important. Participants were then asked to rank themes in order of importance. This was followed by a second round of prioritization with participants being asked to vote on the top ten most important research questions. Participants were allotted 15 minutes to reduce the list of questions to the top five questions they felt were most important. To accomplish this process of data/research question reduction, participants engaged and lobbied one other. Once the final list of questions were agreed upon, themes were refined, as appropriate. The second session concluded with a debrief with participants about their experiences of research question prioritization, The RHE&BJ team, explained next steps regarding data analysis, opportunities related to research dissemination and future collaborations, and they individually shared their expressions of gratitude to participants for their valuable and courageous involvement in the RPAC-ve process.
Although the standard protocol recommends two sessions, there were instances where additional sessions were required. If sessions reached time and there were participants who did have an opportunity to share, the facilitator, CBO partner, and participants agreed to an additional short session to complete prioritization activities.
Synthesis
After the final session, audio recordings and transcripts were reviewed to identify additional questions that were not captured during the session. Similar to Session One, generated questions were separated into, “research questions with answers” and “researchable questions”. The final list of research questions with answers were set aside for later dissemination. Researchable questions were analyzed using thematic analysis to arrive at final list of research questions and priorities.
Report Back to Sites
The list of “questions with answers” were communicated back to the CBO liaisons and discussed with the CBO invited members from Sessions One and Two. These questions were discussed until members who asked the question were satisfied with answers. The project PI referenced sources from peer-reviewed journals, the media, and the Centers for Disease Control and Prevention. Additional resources gathered by the RHE&BJ Core were sent to the liaisons to be sent to the invited community members. RHE&BJ also provided answers to further questions via email provided to the participants.
Discussion
RPAC-ve is an innovative virtual participatory method which allows historically excluded communities to generate research questions and priorities. In collaboration with five CBOs and a national advisory board, we were able to successfully implement RPAC-ve to understand questions BIPOC, queer, pregnant, post-partum, or pregnancy-capable individuals may have about COVID-19. Adapting RPAC to the virtual environment increased accessibility of research participation of groups who were at increased COVID-19 vulnerability.
The RPAC-ve protocol included several key adaptations from the original RPAC protocol. Moving to an online format limited the burden of preparation. Recruitment expanded to social media as well as telephone-based recruitment which expanded geographic reach of the study. This was especially important as there was significant migration out of metropolitan areas during the first two years of the pandemic (Haslag & Weagley, 2023). The online format shifted how sessions were facilitated. Participants were able to contribute by speaking aloud, as well as type messages into the chat function and use Zoom emoji features. These new types of engagement allowed facilitators to gather additional information about the extent to which research questions resonated across participants and develop new questions. The RPAC-ve protocol also included the use of a visual scribe who was tasked with creating a visual representation of sessions. The addition of the scribe allowed the team to capture nuances and connections in the data, as well as create visual products for dissemination. Arts based approaches are being increasingly used in health research (Boydell et al., 2012; Wells et al., 2023). Applying arts-based approaches to research priority setting and may ensure that top priorities are disseminated to a wider range of researchers, policy makers, and funders.
Strong partnerships with CBOs are important for the success of implementing RPAC-ve. Specifically, leveraging existing relationships with confirmation of financial support for their staff time and effort was crucial – even with virtual sessions. In our implementation of RPAC-ve, CBOs directly reached out to participants which ensured they were introduced to the study from a trusted source and several CBO leaders attended sessions which may have helped foster openness between participants and facilitators. Working in partnership with CBOs can help reduce some of the challenges with recruitment and attrition which are common in online research with individuals from lower socioeconomic backgrounds (Lathen & Laestadius, 2021).
Virtual methods pose additional difficulties not observed in in-person data collection. During RPAC-ve sessions a variable number of participants chose to keep their videos off or joined via phone and there was intermittent use of the chat and emoji icons. This limited facilitators’ ability to discern participant emotions and level of engagement, a challenge common in digital research (Slingerland et al., 2022). Studies using online focus groups often face challenges with participants using smart phones to join discussions while in transit, in public spaces, or while multitasking (Lathen & Laestadius, 2021). To try and increase engagement during RPAC-ve sessions, facilitators made active attempts to engage participants both verbally and through chat. The virtual format also required active moderation of both verbal and chat functions. At times the conversation occurring in the chat moved at a quicker pace and captured different topics than the verbal conversation. Conducting RPAC-ve with multiple experienced facilitators can ensure that all participant perspectives are being included.
RPAC-ve protocol may be challenging for participants with limited digital literacy and access to technology. Digital inequities are pervasive in low-income and communities of color. While overall rates of internet usage are high in the U.S., 29% of Black households and 35% of Latinx households do not have broadband internet in their homes compared to 21% of white households. Additionally, Black, Latinx, and lower income households are more likely to rely on smart phones for internet connectivity (Pew Research Center, 2021). While RPAC-ve can be implemented with laptops or smart phones and tablets, there may be differences in participation. Individuals using a smart phone compared to a laptop may experience unstable internet connections, have concerns about using all their data, and may possibly limit their ability to easily use some of the interactive functions like the chat and emoji features. This may lead participants to feel frustrated or excluded, which may lead to disengagement. To address some of these challenges, RPAC-ve implementers may consider strategies such as offering workshops about their selected platform prior to RPAC-ve sessions and compensating participants for attending, providing hotspots or access to laptops and smart devices, and ensuring that each session has a moderator who is adept to identifying and supporting participants who have lower digital literacy (Espinoza Vásquez & Santiago Ortiz, 2023; Lathen & Laestadius, 2021).
Moving to a virtual format does eliminate some of the supports that make participation in RPAC more feasible for participants. RPAC sessions often include childcare and food to ensure that participants can attend and remain engaged during the entirety of sessions. We implemented RPAC-ve during various stages of COVID-19 restrictions where childcare was greatly reduced, and parents were often challenged with navigating online schooling with their children. Parents often had to go off camera or leave temporarily to take care of their children. To address this challenge, RPAC-ve sessions were scheduled outside of school hours and mealtimes. Additionally, moderators shared at the beginning of the session their flexibility with participant’s caregiving and other demands and would check-in with participants who were disengaged for extended periods. Altman and colleagues (Altman et al., 2020) adapted RPAC to a one-day protocol to make participation in RPAC more accessible for pregnant and parenting individuals with opioid use disorder. Participants found the one-day protocol more acceptable and feasible. RPAC-ve is designed to be a flexible approach to serve the contexts of communities disproportionately impacted by health inequities.
Experiences from the development and first implementation of RPAC-ve offer several avenues for future research. Given the active use of the chat during RPAC-ve sessions, researchers should explore research prioritization methods which can be conducted asynchronously such as group texts or online forums. Asynchronous online focus groups have been implemented widely in health research and offer additional benefits of anonymity and removal of the time pressure of responding during synchronous focus groups (Neo et al., 2022; Williams et al., 2012). Researchers should also explore the feasibility of implementing RPAC-ve using various virtual platforms (e.g., GoogleMeet, MicrosoftTeams, Skype) and over the phone to compare usability and participant engagement. Such exploration may inform the implementation of RPAC-ve in various geographic contexts where technological access and digital literacy may vary. Although RPAC-ve has only been implemented in the U.S., research prioritization methods (Graham et al., 2020) and virtual qualitative methods have been implemented globally (Akyirem et al., 2023), thus RPAC-ve may be used in other settings. Finally, future research should compare RPAC-ve to in-person RPAC to assess differences in research questions generated, participant engagement, and satisfaction.
Researchers who wish to implement RPAC-ve should consider some of the limitations of the approach. In addition to aforementioned challenges with digital access and literacy, online methods are subject to issues of privacy. It was unclear whether participants who were off camera or speaking less during sessions were doing so due to concerns about being overheard or other demands in their household. Strategies to try and enhance privacy may include offering transportation and other supports so participants can go to places in their communities with private rooms such as libraries or providing headphones coupled with encouraging use of the chat so participants are not overheard by other house members. RPAC-ve requires more active moderation and potentially more facilitators compared to RPAC. While RPAC-ve can be implemented with a minimum of a CBO staff member liaison, one facilitator, and project staff member, additional facilitators may be necessary depending on the number of attendees and activity in the chat.
Conclusion
In our work we demonstrated how a virtual research prioritization method can be used to involve impacted communities in setting research, healthcare, and policy priorities to inform COVID-19 public health response. Adapting to the virtual environment allowed for new ways of engagement, flexibility during changing public health restrictions, and opportunities to reach communities who were disproportionately impacted by the pandemic. The RPAC-ve protocol can be tailored to fit the needs of other impacted communities and to expand the geographic reach of research prioritization studies.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Robert Wood Johnson Foundation (A135564), Tara Health Foundation (7029992) and Yellow Chair Foundation (7030253). Margaret W. Gichane was supported by National Institute of Mental Health (K01MH134775). Ifeyinwa V. Asiodu was supported by an NICHD/ORWH-funded K12 (K12 HD052163). Brittany D. Chambers was supported by a National Institute on Minority Health and Health Disparities (K01MD015785).
