Abstract
The methodological purpose of this article is to generate practical guidance for researchers studying complex phenomena through an adaptive case study-mixed methods (CS-MM) design. We describe CS-MM design adjustments made in response to our rapidly changeable research conditions that make complex phenomena challenging to study. We leverage Guetterman & Fetters’ (2018) CS-MM design recommendations while discussing specific adaptive design practice areas and advancing new insights gleaned from an expanded 21-month timeline when compared with a previous 4-month CS-MM study. We draw upon Mike Fetters’ extensive scholarly contributions while striving to continue his legacy for a better world and embodying Mike’s “kind mentoring” approach in our team-based study of the complex phenomena involving the response to a public health emergency.
Keywords
Recognizing the inherent complexities of our research settings and adapting the requisite integration of qualitative and quantitative perspectives requires new ways of designing case studies-mixed methods (CS-MM) research. Adaptive design practices represent a departure from implementing predetermined integration procedures because CS-MM researchers assume, from the onset, that their initial plans will adapt to arising conditions. In this work, we draw upon Guetterman and Fetters’ (2018) essential design distinction of CS-MM as case studies nested within mixed methods designs and follow their recommendation to clearly document integration details in our illustrative adaptive practice area descriptions. This documentation is vital to maximize the value of the novel insights derived from the integration within CS-MM research designs. The methodological purpose of this article is to generate practical guidance for researchers navigating and documenting the study of complex phenomena through an adaptive CS-MM research design.
Complex phenomena are perplexing to study because they defy simplistic analyses of cause and effect and involve numerous interrelated and often unknown complex adaptive systems (CASs). That CASs interact with one another in unpredictable and nonlinear ways is well documented as a paradoxical challenge and opportunity (e.g., Lewin, 1993; Weaver, 1948). A growing body of literature elucidates the need for adaptations and re-emphases agility in mixed methods designs (e.g., Creswell & Plano Clark, 2023; Maxwell & Loomis, 2003; Plano & Ivankova, 2016; Poth, 2018). Yet, illustrative guidance for successfully managing the required integration where the whole is greater than the sum of its parts remains a pressing need (Sancartier, 2020).
Adaptive Case Study-Mixed Methods (CS-MM) Research Practice Areas Mapped Onto Design Recommendations.
Leveraging Case Study-Mixed Methods Design Recommendations for Studies of Complex Adaptive Systems
Guetterman and Fetterss’ (2018) review and discussion of mixed methods case studies published between 2011 and 2016 yielded significant contributions, culminating in their advancement of eight design recommendations. Their description of four illustrative examples adeptly highlighted the design options available to researchers that can “help investigators better harness the value added from mixed methods case studies” (p. 912). Key among the design options involved distinguishing two approaches to the requisite integration of qualitative and quantitative research within a case study design. Guetterman and Fetters (2018) then drew attention to the “general paucity of mixed methods features” (p. 913) revealed by their review and pointed to recent mixed methods research practices such as the necessity of a detailed systematic integration focus within the mixed methods research design and the use of joint displays to provide evidence of integrated outcomes. Of particular note for our current discussion focused on adaptive mixed methods case study practices, Guetterman and Fetters (2018) discussed the need for adaptations of case study or mixed methods approaches as a key consideration and specified conventional practices relating to levels of analysis, sampling, and multiple data sources. In our current work, we found four of Guetterman and Fetters’ (2018) recommendations for designing mixed methods case studies (i.e., rationales, boundaries, approaches, and reporting) helpful for discussing our adaptive CS-MM research practice areas.
In Table 1, we demonstrate our mapping of those four design recommendations (Guetterman & Fetters, 2018) onto four adaptive practices initially advanced by Poth (2018) and adapted for our current purposes. Note that an additional two adaptive practice areas (i.e., conditions and expertise) were not highlighted by Guetterman and Fetters’ (2018) review and, thus, beyond the scope of the design recommendations and not discussed. This article examines the extent to which Guetterman and Fetters’ (2018) recommendations are applicable for mixed methods-case studies of complex phenomena, a topic that has not been previously discussed.
Case Conditions for Integration
We begin with the question: What study conditions indicate a complex phenomenon that benefits from a CS-MM design? Our initial conceptualization of the study phenomenon, the public health response to the COVID-19 pandemic as a CAS, provided a first step towards recognizing and capturing its inherent complexity. In the four-month onset CS-MM study (March – June 2020), we used the three key concepts of a CAS to describe our emerging understandings of the public health response to the global COVID-19 pandemic as an unprecedented and unpredictable disruption to the ways people around the world lived their daily lives and interacted with one another, including in the Canadian province of Alberta (for full description, see Poth et al., 2021). During the 21-month extended timeline CS-MM study (March 2020 – December 2021), we generated a more complex representation of how the public health response (a) evolved as an emergent property in response to new understandings about the virus characteristics, for example, transmission rates and variants, (b) influenced and was influenced by various interdependent components, for example, informing and regulating, and (c) adapted to reflect up-to-date information, for example, public health measures and community behavior (see Figure 1). In the sections that follow, we describe how the case conditions met what Poth (2018) describes as the five necessary research conditions for high complexity that, in turn, required adaptive mixed methods research practices across five integration-focused areas of purposes, boundaries, expertise, procedures, and evidence. Interdependent components of public health communication in response to the COVID-19 Pandemic (December 2021).
Case Purpose for Integration
We ask the question: What purpose for innovation might be suitable for studying complex phenomena as a case study nested within a mixed methods design? Our focus on communications during an unprecedented public health crisis represents a research problem requiring an innovative approach to understanding the many social and public health factors that interact and have yet to be studied. We were uncertain about how to begin, yet we were informed by the well-established importance of effective health communication at the onset of public health crises because people seek credible sources of information to understand health risks and ways to cope with them (Austin et al., 2021). Specific to the COVID-19 global pandemic, we recognized a vital component of the early public health response involved the use of regular publicly accessible communications led by regional, national, and international health authorities and government bodies (e.g., Gardner et al., 2021; Wang et al., 2020). Over time, emerging evidence pointed to the benefit of trustworthy sources of information in promoting COVID-19 prevention behaviors and improving vaccine uptake (e.g., Latkin et al., 2020; Malecki et al., 2021). The growing prevalence of the infodemic, where false information and non-rigorous recommendations were spreading through social media, intensified the need for the credibility of data and the trustworthiness of messaging (Mheidly & Fares, 2020).
Shifts Across the Purposes for Integration and Research Questions of the Two Onset and Extended Timeline Studies.
Case Boundaries for Integration
We consider: What boundaries might be suitable for what is currently known about the nested systems of the complex phenomena? During the onset study, we recognized the nested systems of our case as the province of Alberta within the country of Canada. In so doing our case boundaries became focused on the Alberta population, representing approximately 11% of Canada’s total population, with a smaller than average proportion of residents over age 65 (13% compared with 17.5% nationally; Statistics Canada, 2019) and the provincial health system overseen by the Government of Alberta (Ministry of Health) with a Chief Medical Officer of Health (CMOH) serving as the primary spokesperson. Since 1962, all essential basic health care needs have been delivered through the ten provincial and three territorial systems, which are nested within Canada’s publicly funded health care system and governed by the Canada Health Act adopted in 1984 (Statistics Canada, 2020). While each province holds jurisdiction for public health communications, the federal and provincial public health structures create unique opportunities for coordination; the Public Health Agency of Canada was established in 2004 to provide oversight and communications at the national level and is led by the Chief Public Health Officer of Canada (2020).
How we went about generating insights through a cross-analysis approach changed in response to the arising case conditions and emerging understandings of boundaries. In the onset study, we used text mining to identify six key fluctuations in sentiment messaging and word counts and describe how trust was built in public health communications during the first four months (for full description, see Poth et al., 2021). Then, we provided evidence of attending to the emergence, interdependence, and adaptation of topic areas and message contents related to key areas of effective public health communications and risk assessments. In comparison, our unit of analysis shifted from key fluctuations to waves in the current extended timeline study. We defined the wave duration, as represented in Figure 2, by looking at case statistics using downloaded datasets shared by the Government of Alberta via its interactive COVID-19 data app (Government of Alberta, 2021) to identify peaks and areas of stability and confirmed our date intervals by looking for alignment with news reports of the provincial response. As previously discussed, our research questions (see Table 2) demonstrate the shift from using key fluctuations to examine how public health briefings build credibility and trust in the onset study to the focus on maintenance during the subsequent three waves. The assumption that understandings of suitable boundaries emerge is to be expected in CS-MM research that studies complex phenomena. The need for differing expertise to emerge and for teams to form over time in response is to be expected in CS-MM research that studies complex phenomena. Wave definitions for the current expanded case study-mixed methods Study of the COVID-19 pandemic in Alberta, Canada based on case count data. Note. The Y-axis represents case counts and the X-axis the date. Case count data from the Government of Alberta via its interactive COVID-19 data app (Government of Alberta, 2021).
Case Expertise for Integration
Team Member Contributions to Case Expertise for Integration.
Case Procedures for Integration
We explore how our shift to focus during the current extended timeline study on a cross-analysis of waves created the conditions for our use of the metaphor of braided ropes to emerge. We began, as depicted in Figure 3, with an analysis of within-wave changes in message sentiment and topics across public communications using natural language processing techniques of sentiment analysis, word clouds, and topic modeling (Bengfort et al., 2018). We used descriptive statistics to generate frequencies about the use of the two formats by the CMOH to communicate that emerged over time (i.e., Live briefings and Twitter) and explored how public health communications shifted across waves with provincial case statistics (Government of Alberta, 2021) and patient-level data was aggregated by date to calculate the total number of daily confirmed cases, deaths, hospitalizations, and intensive care admissions. To study local media coverage of key changes in public health measures and public perceptions, we created a timeline from the online news searches of provincial news sources. Our within-wave integration procedures were guided by a qualitative dominant crossover mixed analysis strategy (Hitchcock & Onwuegbuzie, 2020). The braided metaphor was essential for generating our case meta-inferences derived from our cross-analysis of four waves. Shifts in case procedures and approaches to integration are to be expected in CS-MM research that studies complex phenomena. Current expanded timeline study within-wave integration procedures and use of braided ropes metaphor. Note. CMOH = Chief Medical Officer of Health.
Case Evidence of Integration
Recent practice discussions involving researchers from around the globe (e.g., Fàbregues et al., 2021; Guetterman et al., 2022; Makabe et al., 2022) has informed our thinking about quality criteria and integration evidence. As a result we think more broadly about the types of integration evidence associated with CS-MM designs. None of our procedures reflect established strategies to generate meta-inferences. In the onset-focused study, we derived our case meta-inferences by attending to three key concepts of complex adaptive systems: emergence, interdependence, and adaptation across fluctuations within the first wave. Our detailed descriptions of emerging data trends to inform risk assessments, demonstrating care and openness in describing a range of concerns and accessible supports, and being specific about the actions of individuals affecting others was a reasonable start to providing evidence of validity for our onset CS-MM study meta-inferences (Poth et al., 2021). The current expanded timeline study provided the opportunity for generating further meta-inferences reflective of the contemporary descriptions of and expectations for effective public health communications that are more realistic of the rapidly evolving and unpredictable conditions in which a global pandemic unfolds over time across waves. Specifically, our descriptions of defining the wave durations were detailed, as were our accounts of the processes involved in collecting and analyzing evidence for the integration of each case. Among the key insights was identifying an interdependency among various information sources. These sources collectively influence each other in the complex landscape surrounding the COVID-19 pandemic, as depicted in Figure 4. The three information sources continue to exert influence and serve as integral evidence for case integration, provide insights into the characteristics of the cases, and inform the development of meta-inferences. Interdependencies among the information sources informing the study of adjustments in public health pandemic communications. Note. CMOH = Chief Medical Officer of Health. Source: Adapted from Figure 1 (Poth et al., 2021).
Our generation of holistic and timely cross-wave case meta-inferences would have been inaccessible by either qualitative or quantitative data sets alone, and the integrated case descriptions of individual pandemic waves benefited from the use of text mining to efficiently manage the consistent analysis of large volumes of freely accessible qualitative and quantitative datasets. The pandemic provided unprecedented access to open datasets that could profoundly impact public health responses to crises in the future and required new ways of thinking about quality criteria and evidence of integration. Our understanding of appropriate evidence of integration to emerge is to be expected in CS-MM research that studies complex phenomena.
Contributions to Mixed Methods Research
This article contributes much-needed practical guidance for CS-MM researchers navigating and documenting the study of complex phenomena through an illustrative study and discussion of six integration-focused adaptive design practice areas. To that end, we heed the calls by Guetterman and Fetters (2018) to make explicit the requisite integration in our CS-MM designs and offer novel opportunities for visualizations to juxtapose information (Tables 1, 2, and 3) and data (Figure 2), and display emerging understandings (Figures 1, 4) and integration procedures (Figure 3).
Together, our discussions of adaptive CS-MM research practice areas and the use of visualizations benefit CS-MM researchers’ study of complex phenomena by attending in new ways to the three key characteristics of CAS: emergence, interdependency, and adaptation. By identifying the case conditions for integration as involving the study of a complex phenomenon and CS-MM as a suitable design, we provide methodological guidance for a design that realistically represents and accounts for many of the multiple interacting influences within the rapidly changeable research conditions we encountered. By framing the case purpose of CS-MM for integration as emergent and not dependent on the problem being well understood, we offer researchers who study complex phenomena freedom from guiding literature and methodology. By describing what is initially known about the interrelated and nested systems and influences of the CS-MM boundaries for integration, we can promote the assimilation of emerging understandings in further boundary considerations. Conceptualizing evolving public health communications as CASs offers a lens for studying complex public health phenomena using a holistic focus on waves within a CS-MM design that departs from examining their parts in isolation to allow attending to system interdependencies and dynamic influences on and by one another. By forming the case expertise for integration in responsive, integrative, and emergent ways, we make the leveraging of existing team expertise possible in new ways. In turn, this helps foster innovations in researcher combinations and methodologies involved in CS-MM designs. By anticipating case procedures for integration as agile and implementation not required as initially planned, it can stimulate innovations in CS-MM designs such as the use of open datasets. Normalizing the need for procedural adaptations helps us reconsider stability assumptions of the research conditions across time. Assessing the quality of case insights on practical merit and not only on established expectations of methodological rigor can inspire innovations in CS-MM research evidence when assessing the validity of outcomes.
Strengths, Limitations, and Future Directions
Open datasets, freely available for download from the Internet, are recognized for their ease of timely access to valuable yet limited information for the current study of the evolving public health communications during the COVID-19 pandemic in Alberta, Canada. Our study findings should be considered in light of our geographical focus on the COVID-19 pandemic in Alberta during the specified timeframes and by the open data sources selected for inclusion. The public health communications in Alberta are likely influencing and being influenced by communication efforts in other provinces within Canada and may be vastly different from other countries. Further research would be necessary to test the veracity of these open data sources in contexts beyond the current study and to seek open data sources beyond what was included in the current study.
Footnotes
Author Contributions
We dedicate this paper to Mike Fetters in recognition of his numerous contributions as a scholar, clinician, and friend. Mike was indeed a giant in the field of mixed methods research who was generous with his time and energy, and we are all better for it. His legacy lives on in the work he completed and the people he inspired. Cheryl speaks from experience, it was his gentle nudge and words of encouragement at dinner during the 2016 MMIRA global conference in Durham, UK that helped her see through to publication her book that became Innovations in Mixed Methods Research (Poth, 2018).
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Poth serve as an associate editor for JMMR.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: We gratefully acknowledge funding for the study from the University of Alberta’s Endowment Fund for the Future - Support for the Advancement of Scholarship Program.
