Abstract
Historically, qualitative research has complemented quantitative biologic and epidemiologic studies to provide a more complete understanding of pandemics. The COVID-19 pandemic has generated unique and novel challenges for qualitative researchers, who have embraced creative solutions including virtual focus groups and rapid analyses to continue their work. We present our experience conducting a multilingual global qualitative study of healthcare resilience among teams of pediatric oncology professionals during the COVID-19 pandemic. We provide an in-depth description of our methodology and an analysis of factors we believe contributed to our study’s success including our use of technology, engagement of a large multilingual team, global partnerships, and framework-based rapid analysis. We hope these techniques may be useful to qualitative researchers conducting studies during the current pandemic, as well as for all pediatric oncology studies including multiple languages or geographically disparate subjects.
Introduction
The COVID-19 pandemic has changed how individuals interact with their communities, how patients interact with healthcare systems, and how investigators conduct research. While many of the initial studies on COVID-19 reported quantitative analyses, more recent qualitative work has explored the perspectives of healthcare providers (Liu et al., 2020; Munawar & Choudhry, 2020) and the impact of the pandemic on patients with chronic diseases (Gharzai et al., 2020; Giebel et al., 2020), including pediatric cancer (Graetz et al., 2021a, 2021b, 2021c; Saab et al., 2020; Vasquez et al., 2020). During previous pandemics, qualitative research has complemented quantitative biologic and epidemiologic studies to allow for a more complete understanding, particularly exploring social responses (Teti et al., 2020). However, there are unique challenges to conducting qualitative research under those circumstances.
Classically, qualitative methods rely on personal data collection including direct observation, interviews, and focus groups. Early in the course of the COVID-19 pandemic we learned the importance of social distancing to reduce the spread of the virus. To continue conducting qualitative research safely, researchers have adapted and emphasized virtual data collection (Dos Santos Marques et al., 2020). Thus far, these methods have been more frequently utilized in high-income countries that have consistent internet access and easy access to technology. Following data collection, rapid analysis has been embraced to share qualitative COVID-19 related study results in a meaningful and timely manner (Vindrola-Padros et al., 2020). In August 2020, we began a study to evaluate the impact of the COVID-19 pandemic on the care of children with cancer worldwide and successfully employed both techniques, including virtual focus groups in low- and middle-income countries, to accomplish our objective. By December 2020, we had completed 19 focus groups at 16 institutions in 16 countries using eight different languages. The speed with which we conducted this study enabled real-time reflection and dissemination of results during an ongoing pandemic. Here we present a detailed account of our study methodology and describe important lessons learned to guide qualitative researchers working during this pandemic and in the future.
Methods
Research design and participant selection
This work was conducted through the Department of Global Pediatric Medicine at St. Jude Children’s Research Hospital (SJCRH). It was the second part of sequential multiple methods study that sought to describe the impact of the COVID-19 pandemic on pediatric cancer care globally (Graetz et al., 2021a, 2021c). Participants for the qualitative cohort were purposefully selected based on data collected through a cross-sectional survey, the results of which have been previously published (Graetz et al., 2021a). The final survey question asked participants “would you like to tell us more about your institution’s experience during the COVID-19 pandemic?”. Only participants who answered “yes” and provided an email address were eligible for inclusion in the qualitative sample. Over 100 respondents met inclusion criteria. Of these, we purposefully sampled to include individuals from institutions that represented varying geographical contexts and resource settings. The World Bank assigns the world’s economies to four income groups (low, lower-middle, upper-middle, and high-income countries) and all were represented in our sampling (World Bank Country and Lending Groups—World Bank Data Help Desk, 2022). Furthermore, all regions as defined by the World Health Organization (“WHO | Definition of Regional Groupings,” 2017) (Americas, European, African, Eastern Mediterranean, South-East Asian, and Western Pacific regions) were included. Using answers from the survey, we selected institutions with a large volume of pediatric oncology patients (>50 new pediatric cancer patients/year) whose capacity to provide quality cancer care, including access to treatment modalities and changes in patient volume, had been impacted by the pandemic as defined by the quantitative assessment. Ultimately, we selected 16 different institutions (Figure 1). Map of included institutions. institutions at which focus groups were conducted are included, with countries identified by world bank income level.
Ethics
The institutional review board at SJCRH reviewed and approved the study with SJCRH identified as the coordinating center. Additional review and approval was conducted by regulatory bodies and ethics committees at individual sites as required. All participants provided verbal consent for participation and recording.
Local principal investigator and focus group selection
Focus group participants.
Focus group logistics
Focus groups were conducted in the official language of the participating country to ensure participants would be able to fluently express their views and opinions without difficulty. This approach resulted in focus groups conducted in eight languages (Table 1). Due to considerations for time zone differences and work schedules during a pandemic, the team at SJCRH was flexible, and meetings were scheduled around participant availability, including weekend, evening, and early morning hours as we worked across time zones. Once participants were selected and invited, groups were created on a commonly utilized text-messaging application (Whatsapp) for each focus group to facilitate coordination and real-time troubleshooting day-of connectivity issues.
All focus groups were conducted virtually using an online video conferencing platform (Zoom or WebEx). For most focus groups, participants joined through individual links from their homes or offices to facilitate compliance with local social distancing recommendations. At some institutions, particularly in regions with inconsistent internet access, participants were physically together in a large conference room.
Participants were informed before the focus group that they would be asked to turn on their video cameras if possible, and we encouraged everyone to use the “gallery” or “grid” view so that they could see one another. This allowed for face-to-face communication, that in many ways was as intimate as an in-person focus group in which everyone is at the same table but not necessarily able to make eye-contact. We avoided using the share screen feature, even when reviewing focus group ground rules, to encourage participants to focus on one another rather than a presentation. The written chat function of the video conferencing platform was used for technology issues only to encourage fluid dialogue and enable us to capture all relevant focus group data on audio recording. One facilitator was tasked with troubleshooting technology concerns as they arose while the other focused on the study questions and group dynamics. In a few instances, participants without access to a microphone utilized the chat function to enter their responses, which were read out loud by a facilitator to capture the data on the recording.
Data Collection
A semi-structured focus group guide (Supplementary Material) was created based on seven factors previously described as important for resilience in healthcare (Iflaifel et al., 2020). These factors included: teamwork, in-situ practical experience, exposure to diverse views on patient’s situation, trade-offs, protocols and checklists, system design, and work arounds. The guide was written in English and iteratively revised by the investigator team. This guide was translated into Spanish and then back-translated and reviewed by bilingual members of the research team. Pilot focus groups were held in Spanish and English, and the interview guides were revised based on feedback from piloting. After this process, the English guide was translated into the additional languages by bilingual members of the research team. As available, a second bilingual member of the research team checked each translation.
Each focus group was moderated by two facilitators. Three core members of the research team conducted focus groups in English and Spanish. Additional facilitators who were native speakers of the languages needed for the non-English and non-Spanish focus groups were recruited from SJCRH. While the Department of Global Pediatric Medicine at SJCRH has many bilingual native speakers, we were concerned that pre-existing, sometimes hierarchical, relationships between department staff and study participants would affect comfort and honesty during the focus groups. We thus recruited bilingual staff from other departments at SJCRH. Our study team emailed a SJCRH listserv of postdoctoral fellows and approved interpreters asking for volunteer bilingual native speakers in six additional languages (Portuguese, French, Arabic, Mandarin, Russian, Bahasa Indonesia). Where possible, we recruited two facilitators for each language. Many recruited facilitators were bilingual and from the country in which the focus group was hosted, allowing for not only language but also cultural competence. Most volunteer facilitators had not previously conducted qualitative research. To address this, we organized formal facilitator training in which we briefly introduced qualitative research, reviewed the theory behind our study, and described the planned study methodology. This training was conducted virtually on the same video conferencing platform used for focus groups and included observed simulated practice facilitation Facilitators also attended pilot focus groups as observers. Following each focus group, volunteer facilitators were sent a link to an online survey to elicit feedback on their experience.
Focus groups were scheduled by local PIs in collaboration with the SJRCH investigator team. Each focus group was scheduled for 2 hours. Facilitators introduced themselves, set ground rules for the discussion that encouraged respect and diminished hierarchies, and had participants introduce themselves before beginning with the study questions. The shortest focus group lasted 45 minutes; most lasted 2 hours. Virtual focus groups were audio recorded. Audio-recordings were professionally transcribed and translated into English. Translated transcripts were reviewed and compared to the audio-recordings by bilingual members of the research team to ensure clarity and accuracy of translation. Focus groups were conducted from September through mid-October 2020 with transcription and translation beginning in mid-September. A timeline for the entire study process is depicted in Figure 2. Study timeline. timeline for study process from site selection through manuscript preparation.
Data analysis
This study was designed with a plan to utilize rapid turn-around analysis and a specific framework was used to structure the focus group guide and analysis templates. This framework was derived from the resilience in healthcare literature (Iflaifel et al., 2020) and structured around seven factors that have been demonstrated to facilitate resilience. At the time of study inception, the early qualitative work on COVID-19 focused largely on experiences and perspectives of healthcare professionals. We chose a resilience framework and utilized focus groups with the intention to explore positive systems-based solutions employed by teams of childhood cancer professionals across the world, in the hopes these expressions of resilience might be shared and adapted between settings. In addition, as topics raised had the potential to be sensitive to participants, we considered trauma-based interview techniques (Biddle et al., 2013) when designing our discussion guide.
Factors, domains and associated interview questions.
Lessons learned
The research strategy employed in this study was successful for quickly studying the perspectives of interdisciplinary pediatric cancer teams around the world during a pandemic. Our experience provides continued support for conducting qualitative research during this pandemic and demonstrates the potential effectiveness of virtual data collection and rapid analysis. Furthermore, we demonstrate success utilizing these strategies to quickly conduct global research in settings with limited resources and in many languages simultaneously. This work was enabled through our use of technology, the ability to engage a large multilingual team for conducting focus groups, global partnerships, and a framework-based rapid analysis focused on resilience in healthcare.
Use of technology
We found video conferencing to be an effective way to conduct international focus groups. Virtual focus groups have been prioritized by other qualitative researchers during this pandemic (Dos Santos Marques et al., 2020). In planning our approach, we had initial concerns regarding bandwidth and connectivity in resource-limited participating centers. Ultimately, technological and internet access issues did not significantly impact our data collection. Part of our success was due to preparation and flexibility. Specifically, local PIs coordinated with the clinical research associate at SJCRH to organize the focus group at a mutually convenient time.
The video conferencing platform we chose (WebEx) was one our research team was familiar with, and which was accessible to all participants via a shared access link. This platform was easily accessible from mobile devices and included a phone call-in link which enabled participation for those without computer access, and in cases of internet disruptions. For one focus group, we used an alternate video conferencing platform (Zoom) due to issues accessing the primary platform for participants in that country. Many of our participants had increased their use of web conferencing during the pandemic and this familiarity likely assisted their ability to participate in our study. When deemed necessary by local PIs, video conferencing test sessions were organized prior to the focus group. Careful planning to optimize the features of the video conferencing platforms and utilizing two facilitators with different tasks was key in the successful operation of focus groups.
Engaging a large multilingual team
Multilingual qualitative research can be challenging, time consuming, and resource intensive, but it is essential to improve our understanding of this global pandemic and its impact on pediatric cancer care. To appropriately evaluate a global phenomenon, we engaged with teams around the world and hosted focus groups in eight different languages. For this study, we prioritized language and cultural competence over previous qualitative research experience, allowing us to expand our research team and achieve our study objectives.
Following each focus group, volunteer facilitators were invited to provide their feedback through an online survey. Eight of the nine facilitators completed this survey. Of the respondents, 100% rated the experience as either “very” or “extremely” enjoyable and all said they would facilitate another focus group if given the opportunity. All facilitators were “extremely” or “very” satisfied with the facilitator training, facilitation guide, scheduling and communication with the study team. However, only 38% felt they had all the information they needed prior to the focus group and the rest stated they only had “some” or “most” of the necessary information. Suggested areas of improvement included the need for additional information about participants and sites prior to the focus groups; “I think it would be better to know more about who will participate in the focus groups and also about the hospital that these people work in before the meeting.” Facilitators also made suggestions regarding study design based on their experiences, “Another option could be one-on-one interviews, their technical logistics would be much easier to arrange.” One facilitator wondered about the cultural acceptability of the ground rule of “first-name basis,” suggesting “Other countries are very territorial of titles and ranks and if in the same focus group you have a nurse and a head of department it won’t fly good at all to ask them to be on first name basis.”
Unfortunately, most published research does not provide details regarding translation procedures (Regmi et al., 2010), and there are no standards for conducting or reporting qualitative research in more than one language (Alzyood et al., 2020; Squires, 2009). To maintain rigor, we employed piloting, used professional transcription and translations services, and conducted member checking. These elements allowed for reliability and validity even in the context focus groups being piloted in only English and Spanish. While we were able to collect data in multiple languages by training bilingual facilitators, we did not have the human resources to conduct analysis in multiple languages. We thus had audio-recordings translated and transcribed by a professional service we have worked with on previous multilingual qualitative research (Garza et al., 2021; Graetz et al., 2020, 2021b). To ensure validity, we had bilingual facilitators review transcripts and compare them to audio-recordings. When possible, we had a second bilingual facilitator review the first validation.
We acknowledge that not all investigators will have a large pool of available multilingual facilitators. However, our experience demonstrates that volunteers without prior qualitative research experience can be successfully trained for data collection and that multilingual data collection can be combined with unilingual analysis by a smaller team. We hope this encourages other researchers to aim for inclusivity by creatively harnessing the linguistic skills of partners and collaborators
Global partnerships
In addition to a large team of focus group facilitators, we partnered with local PIs at every included institution. These partnerships enabled us to structure our study around individual institutional needs and processes, and to ensure the study was conducted in a way that was relevant to the participants and culturally appropriate. Our history of successful collaboration fostered trust between researchers at SJCRH and international partners which was essential for rapid data collection during a time where everyone had many competing priorities.
We also maintained a degree of flexibility in our study design which enabled adaptation. We relied on local PIs to determine how many focus groups would be necessary at each institution and who would participate, with the composition of each focus group unique and specific to each institution. PIs helped us understand these needs, schedule focus groups, and determine if participants would join separately from home or together at the hospital via a shared link, depending on local regulations and preferences. Local PIs participated in the focus groups themselves. This eased logistics (e.g., PIs were able to ensure participants had protected time and arrived on time) and encouraged trust between other participants and focus group facilitators. Following data collection, local PIs reviewed the results of our analysis as a form of member checking (Candela, 2019). This also allowed PIs to reflect upon the situation at their own institution and learn from other participating hospitals. These individuals were included on the authorship team and reviewed all manuscripts published based on this work.
Framework-based rapid analysis
We utilized rapid analysis, which enables qualitative researchers to efficiently collect and analyze large amounts of qualitative data (Vindrola-Padros & Johnson, 2020). Given the nature of the phenomenon that we were seeking to describe, resilient healthcare for pediatric cancer care during a pandemic, the selected framework aided in a structured approach for data collection and analysis. Furthermore, the framework template allowed us to produce more concise data tables (Watkins, 2017), facilitating the reporting our findings in a timely matter. The conceptual framework domains in this evaluation were flexible enough to enable analysis within a single institution experience and across institutions with varying resource levels and geographical contexts, similar to prior reports (Skillman et al. 2019). Importantly, the analysis team piloted the template on a subset of transcripts and adjusted the domains slightly to ensure it could be used to appropriately capture and summarize the collected data. Findings from rapid analysis have been demonstrated to be consistent with in-depth analysis (Gale et al., 2019) and can expedite data analysis time (Taylor et al., 2018). Furthermore, this technique has been established as an effective way to conduct global qualitative research during the COVID-19 pandemic (Vindrola-Padros et al., 2020).
In our data analysis, audio transcripts were used due to the diverse set of languages employed for the focus groups and the need to consolidate in a single language (English) for the four team members utilizing the analysis templates. Given the different steps taken to ensure adequate translation for reliability and validity, we do not believe important points were lost by not using video recordings.
Our approach to these focus groups helped participants engage with our study and many local PIs reported that the experience of coming together through the focus group was therapeutic for their teams; “it was the first time the various parties involved in pediatric oncology at the Uganda Cancer Institute got together to exchange experiences...that is thanks to you” (PI from Uganda); “some of the reflections made us see all the work done with other eyes” (PI from Brazil). Matrices, which have been demonstrated to be helpful for structuring interventions with limited resources (Averill, 2002), were used to synthesize and communicate research findings.
Conclusion
Qualitative researchers, like other scientists, have found creative ways to adapt and adjust their work to the realities of the COVID-19 pandemic. We partnered with local PIs who had established relationships with SJCRH which engendered trust and facilitated study adaptation. This process taught us about our own institution’s potential for multilingual qualitative research and the power of engaging a global network to conduct culturally relevant, impactful qualitative research. In addition, we learned that virtual focus groups can be as effective as those conducted in-person. This process was cost- and time-efficient since the need to travel was eliminated. We are hopeful that our experience will inspire other qualitative researchers to consider the possibility of global, multilingual, virtual data collection both during this pandemic and beyond.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Contributorship
DG, CRG, and DCM developed the idea. DG and ES drafted the manuscript and prepared the tables and figures. All authors contributed to data collection, the interpretation of the findings, the editing of the article, and the approval of the final submitted version.
