Reimagining Convergent Designs: Advancing Qualitatively-Driven Adaptive Designs Through Concurrent Analysis Strategies

Abstract

This methodological article advances guidance for implementing qualitatively-driven mixed methods convergent designs. A convergent design includes the concurrent collection and analysis of qualitative and quantitative data, followed by their integration, allowing researchers to examine a research problem by drawing on different methods and perspectives. Despite the widespread use of concurrent designs, few examples illustrate how qualitative approaches can drive the integration process. Reflecting on their efforts to prioritize the qualitative component within a convergent design as researchers new to MMR, the authors identify key lessons about missed opportunities. Their experiential account highlights the value of concurrent analysis and iterative integration while addressing limitations in prevailing design guidance and building capacity for qualitatively-driven convergent designs.

Introduction

Complex problems are characterized by dynamic interactions among interdependent components, unclear or shifting boundaries, and the need for flexible and adaptive research designs to support meaningful understanding and action (Smith, 2024). Mixed methods research (MMR) is increasingly recognized as valuable for studying such problems because it enables the integration of qualitative and quantitative approaches (Mertens, 2014). Among the core mixed methods designs, convergent designs are widely used because they allow researchers to collect and analyze data streams concurrently, and then integrate the results to generate insights that would not be possible through either approach alone (Creswell & Plano Clark, 2025). Despite the design’s widespread use and its flexibility for examining a research problem by drawing on different methods and perspectives, few examples illustrate how qualitative approaches can meaningfully drive the integration process within convergent designs. This methodological article addresses this gap by reflecting on the experiences of researchers new to MMR who conducted a convergent design. Specifically, based on our experiential account, we identify lessons about missed opportunities and discuss how concurrent analysis and iterative integration through a qualitatively-driven approach can strengthen the implementation of a convergent design. In so doing, we offer a practical reference for researchers seeking to learn from these experiences and extend the practical implementation of guidance found in the rich literature on interactive, integrative, and adaptive designs (e.g., Creswell & Plano Clark, 2025; Maxwell, 2012; Plano Clark & Ivankova, 2016).

MMR is increasingly recognized as a promising approach for addressing complex problems in applied fields such as health professions education (Sherbino et al., 2026). Motivated by the growing relevance and untapped potential of MMR to address a complex health professions education research problem described below, our team pursued formal MMR training and implemented a convergent MMR design. As five health-focused researchers (EB, MYA, SC, QN, DB-V) new to MMR, however, we found that prevailing guidance for conducting convergent MMR offered limited practical support for implementing the design within our health professions education context. In particular, commonly taught approaches tended to portray convergent designs as compartmentalized and fixed, with pre-specified points and purposes of integration. This framing did not fully realize the promise of MMR for responding to the dynamic and interconnected features of our study context. Our experiences therefore prompted critical reflection on how qualitatively-driven integration could be more flexibly enacted within a convergent design.

To learn from these experiences, we invited two additional team members (CP and RK), both experts in qualitatively-driven MMR (QDMMR) and qualitative methodology, to join our reflections. Together, we examined how convergent MMR designs are commonly conveyed to researchers new to MMR in methodological teaching and published literature. These portrayals often emphasize structure and pre-specification, with limited attention to the flexibility and responsiveness required when studying complex problems (Creswell & Plano Clark, 2025). While emergent and adaptive approaches are central to qualitative inquiry and familiar to experienced QDMMR researchers, such practices are seldom explicitly taught or clearly articulated in MMR guidance or reporting. Reflecting on this gap in our own methodological learning, we recognized that key principles such as concurrent analysis and iterative design adaptation—hallmarks of many qualitative research studies—are rarely integrated into guidance for those learning to implement core MMR designs, such as convergent designs. As a result, the potential of QDMMR approaches to illuminate complex phenomena is often underrealized.

We recognize that approaches to qualitative research and QDMMR are not universal. In the broader literature, MMR is defined by the integration of qualitative and quantitative strands of the study, rather than by the use of multiple methods that operate in parallel with separate but complementary aims and procedures. In QDMMR, the qualitative component is prioritized, focusing on participant voices and primarily inductive analysis, while the quantitative component plays a supplementary role and may not be intended to stand on its own (Morse & Cheek, 2014; Poth & Shannon-Baker, 2022). QDMMR is often associated with adaptive (Toledo & Shannon-Baker, 2023), emergent, and nonlinear design processes (Poth & Shannon-Baker, 2022), though such approaches are not exclusive to QDMMR. Adaptive designs remain open to revising aspects of the study design as it unfolds, including the framing of the problem, the composition of the research team, research questions, and strategies for sampling, data collection, and analysis in response to preliminary findings or unanticipated challenges (Poth, 2018, 2022). Similarly, Creamer’s (2019, 2022) concept of “fully integrated” MMR and Maxwell’s focus on MMR as a dynamic research process move the field beyond rigid design “types” (Maxwell et al., 2015) and toward MMR practices that are flexible and responsive. Within these approaches, qualitative and quantitative components can interact continuously and dialectically to develop a more complexity-informed picture of a phenomenon and its context (Hall et al., 2008; Maxwell, 2022; Maxwell et al., 2015; Poth, 2022).

Although QDMMR is not inherently adaptive, it is closely aligned with qualitative ways of thinking that resonate with what Braun & Clarke (2024) describe as “Big Q” qualitative research. Big Q research is qualitative research that “embraces ‘researcher subjectivity’ as a resource for research, rather than a threat to be contained, and meaning and knowledge as contextually situated, partial, and provisional” (Braun & Clarke, 2024, p. 2). In this approach, researchers actively draw upon their knowledge and perspectives to shape their research designs and interpret data in ways that respond to evolving understandings of the phenomenon under study and address their research questions. In contrast, little q research refers to studies using non-numerical data while pursuing aims similar to those of quantitative research, seeking replicable and objective knowledge and attempting to minimize researcher influence on the research process. Some QDMMR approaches focus more on the prioritization of qualitative data and analytic procedures in ways that align more closely with little q orientations (e.g., Morse & Cheek, 2014). However, much of the QDMMR literature, and the approach taken in this article, prioritize qualitative epistemologies (Mason, 2006) and theoretical orientations (Hesse-Biber, 2010, 2018) that embrace researchers’ subjective perspectives and the contextually situated nature of data and findings (Toledo & Shannon-Baker, 2023). This qualitative orientation helps explain why practices such as concurrent analysis, iterative interpretation, and responsive integration are particularly important when implementing qualitatively-driven convergent mixed methods designs.

In this article, we share the missed opportunities and lessons learned from our original convergent MMR study in order to support others new to MMR. We begin by describing the research context and our experiences conducting an empirical convergent MMR study, designed to capture the complex emotions and experiences of both faculty and learners during clinical assessments. These assessments, known as Entrustable Professional Activity assessments, occur in dynamic environments where participants must navigate multiple, shifting, and often competing high-stakes priorities, including teaching and learning, professional assessment, and gatekeeping to ensure the safe and effective delivery of both clinical care and education of medical residents (Bandiera & Van Melle, 2020). We then reflect on how our early understandings of convergent MMR design fell short of realizing the potential of MMR for addressing such complex health education problems. Drawing on these reflections, we consider “what if” scenarios that explore how adopting adaptive QDMMR practices might have better supported our complex research questions and generated integrated and actionable results. We further describe the value of practices such as concurrent analysis and reflexivity as key features of adaptive QDMMR, offering examples of how these practices could have been incorporated into our original study. Ultimately, we aim to empower other researchers new to MMR in embracing more flexible and adaptive approaches to QDMMR design, while drawing on well-established practices from the qualitative methodological literature.

To make these experiential insights accessible, we structure this article in four parts. First, we describe the context of the complex problem in health professions education and outline how MMR was understood and applied within the field at the time of our study. Second, we present our convergent MMR study, including our team’s approach, backgrounds, and interactions that prompted our post-study reflections. Third, we present the key lessons that we learned through this reflective process. Finally, we draw on the qualitative methodological literature to offer practical guidance on how to capitalize on qualitative strategies within iterative and adaptive QDMMR designs, centering the concept of concurrent analysis, and describing specific examples of adaptations that often result from concurrent analysis in qualitative research—adjustments to sampling strategies and data collection tools that respond to unanticipated findings.

Our Context: Use of MMR in Health Professions Education

Complexity Inherent to Medical Education Research Context

Emotions are multi-faceted psychological states involving cognitive, affective, motivational, physiological, and expressive processes (Lajoie et al., 2021; Scherer, 2000). In this study, we focus on achievement emotions, which are goal-related, driven by appraisal, and linked to achievement activities (Pekrun, 2022). Because assessment processes influence both educational practice and the delivery of patient care, the interplay of competing priorities related to teaching and learning, assessment and gatekeeping, and patient care adds further complexity. The complexity gets amplified as emotions often run high within these competing demands. Our study therefore sought to better understand the experiences of both faculty educators and learners, and the emotional impact of assessment in clinical training contexts.

At the same time, in medical education, the Royal College of Physicians and Surgeons of Canada has redesigned residency education through the Competence By Design framework, with Entrustable Professional Activity assessments expected to be implemented across all specialties by 2026 (Hall et al., 2020). Although widely adopted, studies have documented unintended consequences, including learner frustration and anxiety, and concerns that an increased emphasis on assessment has, at times, overshadowed learning (Bilgic et al., 2023; Braund et al., 2024; Johnson et al., 2024). As programs are expected to monitor and continually adapt assessment practices, this ongoing change has contributed to a rapidly evolving and complex assessment environment for both learners and faculty (Mirza et al., 2024).

To explore the interplay between emotions and assessment in this complex context, we designed a convergent mixed methods study that gave equal weight to qualitative interviews exploring participants’ emotions and a quantitative survey measuring the emotions associated with Entrustable Professional Activity assessments. We examined emotions related to both the role of Entrustable Professional Activity assessments in learning and to the assessment processes themselves (e.g., who initiates assessments, expired assessments, residents completing forms themselves), as these processes influence how useful Entrustable Professional Activity assessments are for learning. If residents experience persistent negative emotions related to both learning and assessment processes, learning may be hindered.

We planned to integrate the qualitative and quantitative data after completing both strands; however, we encountered contradictory findings that raised more questions than answers. Because our design did not allow for iterative adjustments to recruitment or interview guides, we were unable to further probe these discrepancies. In the sections that follow, we examine our design choices, findings, and the key lessons learned from this experience.

How Is MMR Used in Health Professions Education?

Using MMR to explore the emotions and experiences of different stakeholders in Entrustable Professional Activity assessments helps address persistent gaps in the literature. Much of the existing assessment and Competence By Design research continues to rely on qualitative and quantitative designs in isolation, resulting in struggles both to contextualize quantitative findings demonstrating impact of assessment, and to demonstrate the overarching impact of assessment struggles that have been described by learners, educators, and educational leaders (McKinley, 2015). This separation reinforces the view that quantitative methods are aligned with deductive hypothesis testing and qualitative methods with inductive theory building (Bordage, 2007). In contrast, MMR in health professions education has been positioned as a pragmatic approach that integrates multiple perspectives to generate richer meta-perspectives (Lavelle et al., 2013). Such an approach has the strong potential for generating findings that are both grounded in a rich sense of context and broadly transferable, enhancing applicability in real-world educational settings.

Despite this potential, the health professions education literature using MMR remains limited in both volume and quality (Sherbino et al., 2026). Studies that use MMR most often rely on core design types (convergent, explanatory sequential, or exploratory sequential designs) (Fetters, 2020), and tend to follow relatively structured approaches that rarely adapt or depart from the established design configuration. This may reflect (1) research design textbooks focusing on the three core MMR designs, (2) methodological teaching in health professions education emphasizing the core MMR designs, and (3) the clarity that these core designs provide in a field with limited examples of complex MMR (Lavelle et al., 2013). However, as others have argued (Maxwell et al., 2015), reliance on relatively “fixed” designs of MMR may limit the field’s ability to address complex problems and achieve meaningful integration, as occurred in our study. Additionally, within many health professions education MMR studies, the qualitative and quantitative components are either weighted equally, or the quantitative component is dominant, and integration is often limited because researchers do not adapt their designs to capitalize on unexpected insights gleaned from rich qualitative data and analyses.

Who Are the Team Members?

The original convergent study and the post-study reflections were conducted by a team with expertise across many education and research domains. EB has expertise in assessment in medical education, and led the project and the paper. MYA, QN, and DB-V, clinicians with expertise in medical education, contributed to the original study and paper development. SC contributed expertise in assessment.

As a team, we had (1) significant expertise in educational assessment theory and practice, (2) experience with quantitative and qualitative methodologies and methods in health professions education, and (3) emerging expertise in MMR. However, we did not have team members with a strong Big Q qualitative expertise or experience in QDMMR. At the time, we did not recognize the value this expertise could bring, as our study did not initially aim to center qualitative ways of thinking. We later engaged RK and CP for their qualitative and QDMMR expertise, which led to an “aha” moment about how these approaches could have contributed to our study.

Our Original Convergent Study: Assumptions of Convergent Design Guidance

Among core designs, convergent designs generally involve the concurrent but separate collection and analysis of qualitative and quantitative data, with integration occurring only after analysis of each strand is complete (Creswell & Plano Clark, 2025). This depiction of separate and distinct strands with a single point of integration is reinforced through design diagrams like that of our original study (Figure 1). The structure of the convergent design is particularly appealing for addressing research questions that benefit from comparing or combining insights from different data types collected within the same timeframe. Yet, this standard depiction can constrain researchers’ thinking about how and when integration will take place, and limit creativity and adaptation. Of particular concern for the convergent design involves aligning constructs across qualitative and quantitative research strands to ensure that the data can be meaningfully compared, contrasted, or merged, generally after all data have been generated and often analyzed separately (Fetters et al., 2013), even though the affordance of qualitative approaches is that they are often exploratory and the insights they generate can be unexpected. Thus, alignment cannot always be determined at the outset.

Figure 1.

Original convergent study design diagram

Though the parallel convergent design is central to entry-level MMR resources and is a feasible design for researchers new to MMR, it largely ignores the rich literature advocating for dynamic and adaptive designs that can respond meaningfully to the research context and develop a rich picture of the complex interactions within that context and between different data sources (Hesse-Biber, 2018). These approaches are often portrayed as “advanced” compared to “technically simpler variation[s]” of convergent designs, discouraging new researchers’ from taking up an adaptive approach (Fetters et al., 2013). Adaptive approaches, by contrast, tend to be open ended, and can leave researchers new to MMR feeling uncertain about what kind of thread they ought to follow and how. Thus, core designs are paradoxically invaluable and constraining for researchers. Invaluable as they are widely used, well-documented, and supported by extensive guidance, but they can also be constraining because they are often presented as fixed templates rather than flexible frameworks (Maxwell et al., 2015). In response, we believe that it is essential for training and other entry level resources to begin to emphasize that core designs serve as starting points, not end points, for MMR.

Our Original Convergent Study: Methods, Findings, and Lessons Learned: The Limits of Parallelism

Our study aimed to address the question: To what extent do the type and intensity of emotions captured in the Medical Emotion Scale scores reflect the emotions described as experienced by residents and faculty regarding Entrustable Professional Activity assessment processes? Our integration strategy was merging, since we sought to use our streams to develop new insights that could disentangle the relationships between emotions, assessment, and participants’ unique characteristics and contexts. This study was approved by the Hamilton Integrated Research Ethics Board (HiREB #14686).

The design (see Figure 1) called for the parallel collection and analysis of data with a matched sample of both residents and faculty. Both strands were weighed equally, and at the end of separate analysis, the findings were merged. We decided to conduct the 2 strands in parallel rather than sequentially given the continually evolving policy and practice contexts within which assessments take place, and their potential to shift such that integration would be difficult because of significant contextual change.

Our participants included residents and faculty in pediatrics, general surgery, and emergency medicine. These contexts were selected because they vary considerably in terms of professional culture, as well as context of clinical assessment. Pediatrics and emergency medicine tend to use learner documentation and reports for assessment purposes, rather than only observation, whereas in surgery, assessors tend to be present to witness the activity they are assessing (Bearman et al., 2023). The quantitative arm included conducting a survey that captured demographic information (e.g., gender, specialty, visible minority status, etc.) and the emotions of residents and faculty regarding assessments through the Medical Emotion Scale (Chopra et al., 2024; Duffy et al., 2020). The qualitative arm included semi-structured interviews with residents and faculty (matched sample); they were asked about their emotions related to their assessment experiences. The survey data were analyzed using descriptive and inferential statistics (ANOVA), and the qualitative data were analyzed using reflexive thematic analysis (Braun & Clarke, 2019a, 2019b). For further details about qualitative and quantitative procedures, see authors.

Our single point of integration was post data collection and analysis of both strands. The emotion scale data (quantitative data) were integrated (merged) with the major observations from each qualitative interview to enhance understanding of the resident and faculty emotions. The integrated results are categorized as expanding (qualitative data confirms and expands on the quantitative data), contradicting (qualitative and quantitative data contradict one another), or confirming (qualitative and quantitative data confirm one another). In particular, we were excited to explore how emotions might be experienced differently in different contexts, and to understand how the experiences of participants might contrast with the seemingly “objective” measures of the Medical Emotion Scale.

Results

Our quantitative findings mostly reinforced our qualitative findings, with some contradictions and missed opportunities for further exploration. Tables 1–3 are joint displays for our original study that show the findings from the quantitative and qualitative strands, and the meta-inferences we generated when we integrated the data sets. Although integration mostly confirmed findings from each data set, there were some contradictions that captured our attention.

Table 1.

Meta-Inferences Regarding the Differences in Emotions Based on Specialty

​	Quantitative trends (survey-MES a ) (residents versus faculty)	Qualitative trends (interviews) (residents versus faculty)	Meta-inference b
Differences in emotions based on specialty	Residents: For differences in emotional responses across specialties (general surgery, pediatrics, and emergency medicine), no differences were observed. However, the means for both negative and positive emotions varied across specialties. For negative emotions, the mean and standard deviation were as follows (with pediatrics having the highest negative emotions and general surgery as the lowest). Emergency medicine: M = 2.50, SD = 0.454 General surgery: M = 2.10, SD = 0.321 Pediatrics: M = 2.92, SD = 0.321 For positive emotions, the mean and standard deviation were (with pediatrics having the highest negative emotions and general surgery as the lowest): Emergency medicine: M = 1.66, SD = 0.47 General surgery: M = 2.5, SD = 0.68 Pediatrics: M = 1.97, SD = 0.89 Faculty: The differences in the mean and SD of negative emotions among faculty members were: Emergency medicine: M = 1.87, SD = .198 General surgery: M = 1.63, SD = 0.229 Pediatrics: M = 1.40, SD = 0.162 Positive emotions: Emergency medicine: M = 2.81, SD = .525 General surgery: M = 3.083, SD = 0.606 Pediatrics: M = 2.33, SD = .428	The study revealed differences in how EPA assessments are perceived across specialties. Faculty members in pediatrics and emergency medicine generally showed more positive emotions toward EPA assessments compared to their colleagues in general surgery.	Contradiction: The study presents a potential contradiction between quantitative and qualitative findings regarding emotions across specialties. Although the quantitative analysis did not reveal major differences, the mean differences indicate that residents in pediatrics report higher negative emotions, whereas general surgery residents report the lowest. Regarding faculty members, those in general surgery show the highest positive emotions, while those in pediatrics report the lowest. In contrast, qualitative insights suggest faculty in pediatrics and emergency medicine have more positive emotions of EPA assessments than those in general surgery.

^aMES = Medical Emotion Scale.

^bMeta-inferences: Confirmation (qualitative and quantitative data confirm one another), expansion (qualitative data confirms and expands on the quantitative data), and contradiction (qualitative and quantitative data contradict one another).

Table 2.

Meta-Inferences Regarding the Differences in Emotions Based on Visible Minority Status

​	Quantitative trends (survey-MES a ) (residents versus faculty)	Qualitative trends (interviews) (residents versus faculty)	Meta-inference b
Differences in emotions based on the visible minority affiliation	Residents: Mean negative emotion score was slightly higher for residents with minority affiliation (M = 2.72, SD = 0.309) compared to non-minority residents (M = 2.30, SD = 0.309). No notable difference in positive emotions was observed between minority and non-minority residents. Faculty: Mean for negative emotions was slightly higher for minority faculty (M = 1.70, SD = 0.200) compared to non-minority faculty (M = 1.50, SD = 0.183). Conversely, the mean positive emotion score was higher for non-minority faculty (M = 2.70, SD = 0.49) than for minority faculty (M = 2.59, SD = 0.451).	There were no specific trends apparent regarding differences in the positive and negative emotions of residents and faculty based on minority affiliations.	Contradiction: Residents and faculty who identify themselves as belonging to a visible minority affiliation feel slightly higher negative emotions. Although we did not identify specific trends in the qualitative data, that might stem from the fact that we did not have specific questions to understand the impact of visible minority status on EPA assessment related emotions.

^aMES = Medical Emotion Scale.

^bMeta-inferences: Confirmation (qualitative and quantitative data confirm one another), expansion (qualitative data confirms and expands on the quantitative data), and contradiction (qualitative and quantitative data contradict one another).

Table 3.

Meta-Inferences Regarding the Differences in Emotions Based on Gender

​	Quantitative trends (survey-MES a ) (residents versus faculty)	Qualitative trends (interviews) (residents versus faculty)	Meta-inference b
Differences in emotions based on gender	Residents: Mean for both negative and positive emotions were different with men having more positive and negative emotions compared to women. For negative emotions: Women: M = 2.23, SD = .269 Men: 2.85, SD = .330 For positive emotions: Female: M = 2.00, SD = .76 Male: M = 2.33, SD = .82 Faculty: Mean differences were observed with men reporting more positive and negative emotions compared to women. For negative emotions: Women: M = 1.50, SD = .193 Men: 1.67, SD = 1.33 For positive emotions: Women: M = 2.62, SD = 1.21 Men: M = 2.68, SD = .95	There were no apparent specific trends observed regarding differences in the positive and negative emotions of residents and faculty based on gender.	Contradiction: Residents and faculty who identify themselves as men feel slightly higher positive emotions. Although we did not identify specific trends in the qualitative data, that might stem from the fact that we did not have specific questions to understand the impact of gender on EPA assessment related emotions.

^aMES = Medical Emotion Scale.

^bMeta-inferences: Confirmation (qualitative and quantitative data confirm one another), expansion (qualitative data confirms and expands on the quantitative data), and contradiction (qualitative and quantitative data contradict one another).

First, based on the quantitative analysis, the mean differences indicate that residents in pediatrics report higher negative emotions, whereas general surgery residents report the lowest. Similarly, faculty members in general surgery show the highest positive emotions. However, qualitative insights suggested that faculty in pediatrics and emergency medicine have more positive emotions regarding assessment experiences than those in general surgery. Second, residents and faculty who identified as women or belonging to a visible minority group were found to have slightly higher negative emotions based on findings from the Medical Emotion Scale, but we did not identify a similar trend through our qualitative work.

Achievements and Missed Opportunities

The study generated novel insights into the interplay between emotions and Entrustable Professional Activity assessments through the separate quantitative and qualitative components (Chopra et al., 2024; Johnson et al., 2024). When results were merged at the interpretation stage, we identified potentially important factors shaping this interplay, including specialty, gender, and minority status of a resident and faculty—findings driven primarily by the quantitative component. However, integrating only at the level of findings left important questions unresolved, particularly these contradictory and potentially consequential results. This limitation is significant given prior research demonstrating that individuals of different genders (e.g., men, women, and gender-diverse individuals) experience residency and faculty life differently, including disparities in the respect they receive, career progression, and pay (Barnes et al., 2019; Menchetti et al., 2022). Moreover, variation in how specialties implement Competence By Design programs may have influenced the quantitative trends observed, reflecting local implementation challenges rather than broader cultural attitudes toward assessment.

Unfortunately, our ability to follow up on these contradictions through qualitative strand was limited due to our parallel convergent design. Because parallel convergent designs only call for interaction between strands at the stage of interpretation (Fetters et al., 2013), our survey results did not inform the interview questions or guide qualitative sampling (a strategy often used in emergent designs) and data collection, hence, we were unable to explore the reasons behind these contradictions. A more iterative and interactive approach could have highlighted participant voices and experiences of assessment, and enabled participants themselves to contribute to the interpretations we were able to make, stemming from the integration of our early qualitative and quantitative findings. While the parallel convergent design offered a useful organizing framework, we faced challenges in coordinating concurrent data collection and analysis, and in interpreting potentially contradictory findings as we integrated across the data findings. The separation of qualitative and quantitative components also prevented us from intentionally expanding our survey dataset by recruiting and sampling to investigate these unexpected and potentially important results. In other words, our convergent design, but also any parallel convergent design, can and should be adapted to suit the specific research questions, contexts, and complexities of a given study. It is with this spirit that after the study was done, we conducted post-study reflections that led us to think about a revised version of our initial convergent design, grounded in QDMMR.

Working as a team, synergizing across our different areas of expertise in qualitative, quantitative, and mixed methods research allowed us to reflect critically on our design and recognize opportunities that we missed—insights we would not have reached individually. However, recognizing missed opportunities in hindsight does not necessarily help researchers who are new to MMR to navigate the uncertainty associated with adaptive, iterative, and qualitatively-driven convergent designs. To offer practical guidance, we draw from the qualitative methodological literature to introduce principles of concurrent analysis and design adaptation. We discuss how an adaptive QDMMR design and a concurrent, iterative approach to qualitatively-driven data collection and analysis could have addressed shortcomings in our study design and produced more integrated and meaningful insights. By making these practices more visible, we aim to help other researchers new to MMR feel more confident engaging in iterative and adaptative designs, and less inclined to avoid adaptation due to uncertainty and messages in MMR teaching and entry-level resources that discourage these approaches, such as those that we encountered and took to heart.

A Contribution to Mixed Methods Research: How a Qualitatively-Driven Convergent Design Could Have Addressed Our Missed Opportunities

As researchers new to MMR, we struggled with how convergent designs are often described, as the language can suggest relatively rigid, linear, and predetermined study designs (Maxwell et al., 2015), often centering parallel convergent designs that include a single point of integration at the interpretation stage. We followed what appeared to be a roadmap for convergent design by identifying an integration point, purpose, and strategy. While this guidance was practical, it limited our ability to respond to emerging insights. In retrospect, we would have benefited from guidance that framed integration as continuous and dynamic, rather than occurring at discrete points in time (Creamer, 2022; Maxwell et al., 2015).

We now see value in qualitatively-driven, adaptive convergent approaches that move away from fixed “points of integration” and toward more flexible and iterative processes common in Big Q qualitative research and QDMMR, where concurrent analysis and adaptation are common. In such an approach, the two study components continuously inform one another, allowing qualitative ways of thinking to guide how researchers respond to complexity. This approach would have helped us avoid the limitations of single-point integration that prevented adaptation in our original design. Importantly, this challenge is not limited to convergent designs; it can also occur in sequential designs, where one component follows the other. Sequential designs could also benefit from emergent and iterative design practices such as staged sampling strategies where initial participants inform later sampling decisions or embedding opportunities for adapting sampling at integration points.

Teaching Core Designs Differently

How we teach about and apply core MMR designs should reflect the complex and evolving realities in which researchers work. Many scholars have therefore called for greater emphasis on MMR purposes and integration processes than those described within design typologies (Creamer, 2022; Hall et al., 2008; Maxwell, 2022; Maxwell et al., 2015; Poth, 2022). Such innovation is not only welcomed but necessary.

Drawing on lessons learned from our experience with ‘single point’ integration at the post-analysis phase, we turn to principles of iteration and recursion common in Big Q qualitative research and QDMMR. Specifically, we describe the concrete practices commonly associated with concurrent and iterative data analysis, and offer examples of adaptations that can respond to concurrent analysis, including sampling adjustments (such as theoretical sampling) and adaptations in qualitative data collection tools (such as interview guides). Throughout this discussion, we thread the concept of reflexivity, which is central to rigorous, transparent, and ethical Big Q research (Olmos-Vega et al., 2023). Our goal is to make these adaptive practices more visible in order to support a shift in how MMR is taught and practiced.

Overview of Concurrent Analysis

Concurrent data analysis has long been advocated in qualitative research traditions. In constructivist grounded theory (CGT), a ubiquitous qualitative approach in our field of health professions education and many other applied fields, concurrent analysis is considered a near-mandatory practice (Charmaz, 2025). From a post-positivist or little q perspective, adapting data collection during a study is sometimes viewed as introducing bias and undermining credibility. However, within a Big Q orientation, subjectivity is understood as both valuable and inevitable, and is managed reflexively, with researchers using theoretical and experiential knowledge and perspectives to make informed and transparent research decisions.

For example, in our study, we observed unexpected trends, such as higher positive emotion among participants from surgery. Rather than dismissing these findings, concurrent analysis could have prompted follow-up interviews to explore possible explanations, such as differences in assessment culture across specialties. Without concurrent analysis and adaptation built into the design, however, we were unable to pursue these leads, leaving important questions unresolved and requiring new studies to address them.

Practices of Concurrent Analysis

In qualitative research involving multiple data sources, the idea of fixed “points of integration” is often incompatible with how research unfolds. For example, ethnographers commonly integrate observational and interview data throughout data collection, with each informing the other. Observations help ethnographers notice the practices of culture that may be so normalized as to be invisible to participants, but those practices cannot be understood without an appreciation of how people within that culture make sense of their practices (Munz, 2017; Walford, 2015). Similarly, in QDMMR convergent designs, concurrent analysis creates opportunities for iterative adaptation across components.

In our study, a more flexible and adaptive QDMMR design could have enabled us to adjust our data collection and analysis approaches as we moved through the study, helping us to better capture the complexity and context of the trends and contradictions we observed. Specifically, concurrent analysis would have allowed us to adjust our interview guide and sampling procedures to explore contradictions between our Medical Emotion Scale results and our interview data, including further exploration of participants’ understanding of how their gender, specialty, or visible minority status interact with emotions in clinical assessment. To illustrate what we might have done differently, we developed a revised study design diagram reflecting a more adaptive QDMMR approach (Figure 2). This reimagined design shifts when and how integration occurs, opens space to prioritize participant voices, and embraces iterative cycles of concurrent analysis and data collection. Encouraging such adaptive use not only aligns with the spirit of methodological pluralism (Hall et al., 2008), but also opens new possibilities for innovation, especially in fields like health professions education, where research problems are inherently complex and benefit from thoughtfully integrated approaches.

Figure 2.

Updated study design diagram for future QDMMR convergent study

However, in our experience with qualitative studies, predominantly CGT (Charmaz, 2024), researchers can rarely make the most of concurrent analysis without careful planning. We can find ourselves preoccupied with other things while interviews move forward without meaningful analysis and adaptation. Thus, we offer a few key practices from qualitative research practices of concurrent analysis, particularly drawing on our experience with CGT where concurrent analysis and adaptation is central (Charmaz, 2024). These practices were selected because we believe that they are concrete tools that are not always shared with researchers new to MMR, but are critical to ensuring that time, space, and mechanisms for adaptation exist for thoughtful concurrent analysis and adaptation.

Planning for Concurrent Analysis

In our experience (RK and CP), concurrent analysis requires a plan. Researchers need to intentionally create “gaps” in their data collection (Watling et al., 2017) or “make early stops to analyze what you find along your path” (Charmaz, 2024, p. 1) to allow for thoughtful review of qualitative data, team discussion, and adjustments to the plan, which sometimes require an institutional ethics board amendment. They also need to create time for this analytic work—when, how, and by whom will the data be analyzed during these “stops”? Some advocate for pausing interview data collection after only two-three interviews (Watling et al., 2017). Alongside these planned “stops”, QDMMR research teams will need to discuss and define team roles and decide who will engage in concurrent analysis throughout the study (Fernald & Duclos, 2005). For small teams of two or three researchers, all researchers may engage in this role, but for larger teams, it is unlikely to be feasible for all team members to participate in all aspects of analysis. Additionally, QDMMR researchers will need to plan their meetings, with the potential to adapt if some major issue or opportunity arises (Fernald & Duclos, 2005). In our experience, these meetings are often scheduled well in advance to ensure that the analysis continues to move forward despite busy schedules.

If we had incorporated concurrent analysis in our study, we would have defined specific analytic roles and paused interviews after several early interviews to review early findings. These meetings could have informed adjustments to recruitment, interview questions, and integration strategies, and may have included follow-up interviews or focus groups to explore, adjust, or elaborate these early insights.

Although concurrent analysis requires sustained team engagement and resources over time, it may ultimately be more efficient. In our case, we are now conducting a new study to explore contradictions identified in the first study, requiring additional funding, ethics approval, and recruitment. Prioritizing concurrent analysis earlier may have prevented this duplication effort.

One of the reasons that we planned a convergent study was because of the feasibility issues we foresaw in terms of longitudinal engagement, and we were worried that a longer time frame could create recruitment and analytic challenges with potential changes to leadership, organizational processes, and assessment systems. For example, we find that researchers often allocate human and financial resources to data analysis at a single point in time. We suggest integrating strategies like longitudinal team meetings to move data analysis forward, and longitudinal allocation of research assistant or other team member’s time for data analysis. In some cases, this may involve more resources; in others, it may be more of a redistribution. We note that while it can be daunting to allocate more resources to analysis, our experience demonstrates that prioritizing concurrent analysis on the first attempt could be more efficient.

Responding to Concurrent Analysis

Iterative engagement with data through concurrent analysis may lead to many types of adaptations to the study. In CGT, there is a strong focus on adapting the sampling strategy and adjusting the interview guide to fill gaps in the data and support theory development. In CGT, these fall under the umbrella term “theoretical sampling”, where researchers aim to build on their early insights and fill gaps in their dataset and refine the theory they aim to develop. This can mean adjusting recruitment, re-interviewing specific participants, adjusting data sources to include new data types (e.g., including document analysis or focus groups to build a robust and responsive data set), or adjusting data collection materials (e.g., interview guides or field note templates) to follow up on and explore concepts identified through concurrent analysis (Charmaz, 2024; Draucker et al., 2007). Specifically, we discuss sampling adaptation and interview guide adaptation below, as these strategies are often used effectively in CGT and are directly relevant to the challenges we faced in our initial convergent design. We note that a lot of QDMMR convergent designs are not necessarily aimed at theory development; however, we believe that adjusting sampling strategies is a critical tool for many Big Q researchers, though the aim may be descriptive or interpretive, without building “theory.”

Sampling Adaptation

Adjusting sampling procedures to follow up on findings that seem counterintuitive, including “negative cases” that appear to challenge overall trends in a data set is a key tool to building robust interpretations in grounded theory (Charmaz, 2024; Glaser & Strauss, 1967). Sometimes, researchers might realize that they need more participants from a particular population, or they need to re-interview participants to further explore initial insights that are critical to researchers’ developing interpretations. In QDMMR convergent designs, we see unique potential to adapt sampling to ensure robust qualitative and quantitative data sets. It may be difficult to anticipate at the outset the factors that may deeply influence participants’ experiences, and researchers may find that their initial sampling approaches left out key groups, or that certain populations were only thinly represented.

For example, in our study, our matched sample—where participants completed both survey and interview components—offered a unique opportunity to cross-reference findings and build richer interpretations. However, we had a higher number of participants from some specialties and social identities over others, and as a result, we could not unpack our contradictory findings around gender and specialty in either our qualitative or quantitative strands with our initial, predefined sample. Because we did not intentionally sample for these participant characteristics, we did not achieve a sufficient sample size to provide a clear picture of these results. Had we engaged in concurrent analysis and identified the tensions between qualitative and quantitative strands early on, we could have bolstered recruitment from specialties or identities where we had limited representation, making sense of the tension between our preliminary survey findings related to gender, visible minority status, and specialty, and contradictory findings in our qualitative strand.

Some of these characteristics could have been identified in advance, though their importance to the experience of assessment and the unique aspects of each characteristic are not always apparent. For example, we did not anticipate how specialty context would matter to emotion until we noticed contradictions between qualitative and quantitative strands. We remain unsure about whether positive emotions in surgery reported through the survey are driven by the surgical program’s assessment culture, or by another nuanced reason. Additionally, in the qualitative strand, we could have also conducted follow-up interviews/focus groups with the current or new participants to deepen our understanding of shared experiences related to assessment and emotions based on specialty and social identity.

Regarding sampling and recruitment strategies for the survey, although in QDMMR, the quantitative strand is supplemental, we could still optimize our recruitment and sampling strategies. Hence, in Figure 2, we have also suggested that based on the initial findings from both the survey and interview data (e.g., differences across gender), we could further recruit residents and faculty based on gender, visible minority status, and specialty to ensure that we have an adequate sample to explore the initial differences identified through the two data streams. For example, we wondered if the positive emotions experienced in surgery were influenced by direct observation of resident performance during assessment, which is generally preferred by residents and faculty (O’Connell et al., 2025), as opposed to indirect reports prevalent in other specialties. In retrospect, we could have expanded our sample to include anesthesiology or other contexts in which direct observation is common. Additionally, at the end, once the interview cycles were completed, we could modify the survey questions to explore the unique findings, and recruit a larger sample of participants to expand our understanding of the findings.

Interview Guide Adaptation

Another common approach to adaptation based on concurrent analysis is the adjustment of data collection materials such as interview guides. Adjusting the interview guide ensures that interviewers touch on key topics that researchers may not have been able to anticipate in advance and, at the same time, dispense with directions they thought would be fruitful but that, in practice, led to superficial answers. Interview guide adjustments might be general, changing the protocol or approach so that all future participants are able to speak to the new questions, or they might be targeted, where researchers tailor specific questions that only certain participants might speak to, based on their unique perspectives or experiences.

Our study was guided by our knowledge that different specialty cultures and associated practices of assessment, and social identities impact assessment experiences and emotions, and we were confused when the gendered differences in quantitative results did not bear out in the qualitative findings. However, due to the parallel and inflexible nature of our initial study, we couldn’t unpack the contradicting trends that we observed. In response, a flexible design would have allowed us to adapt our interview guide to include a specific question about how participant assessment-related emotions might have been impacted by their gender, and ask them to provide a concrete example of an instance when their gender impacted their emotions as they were tackling the assessment process.

We note that it is critical for teams to think reflexively about the implications of their decisions to adapt their QDMMR study design—the aim is to explore and challenge interpretations, rather than to generate data that can confirm the team’s preconceptions. In this process, a diverse team with trust is key to ensuring robust discussion and challenging preconceptions, and concurrent analysis meetings should be structured to allow for this type of reflexivity (Olmos-Vega et al., 2023). Additionally, it’s critical that researchers engaging with adaptive QDMMR keep a comprehensive audit trail to track their decisions, documenting how and why recruitment and data collection approaches were adjusted, carefully archiving each version of recruitment materials and interview guides for future use in team reflection and to ensure transparency in reporting (Cutcliffe & McKenna, 2004; Tracy, 2010).

When Is It Enough?

Adaptive designs raise questions about when and where sampling and analysis should stop, without a predefined sample size, and debates about what constitutes “enough” data in qualitative research have a long history (Braun & Clarke, 2019a, 2019b). Ultimately, for Big Q research, most researchers agree that there is no magic number, but that “saturation” or “sufficiency” (Dey, 2003) can look very different with different research aims, questions, and study designs (Braun & Clarke, 2019a, 2019b; LaDonna et al., 2021; Malterud et al., 2016). However, there is always a moment where researchers glean a sense that their findings are coherent and well supported by their data, and new data and further analytic work no longer challenges their working interpretations (Morse, 2010). Checking in with diverse team members and engaging in member reflections to check the resonance of interpretations with participants can help to ensure that findings offer a coherent representation of the phenomenon (Bloor, 2001; Tracy, 2026). To communicate the factors that play into sufficiency, many researchers point to the concept of “information power” (Malterud et al., 2016) as a way to consider how much data may be needed relative to breadth of the study’s aims (e.g., narrow aims require smaller samples), specificity of the sample (e.g., defined study populations may involve less diversity and require smaller samples), use of established theory to guide the study (e.g., theory guides sampling and data collection to allow for targeted sampling and data collection), data quality or richness (e.g., more data are required when data are thin, vague, or tangential), and the analysis strategy selected (e.g., descriptive analyses may require less data than more theoretical approaches).

In QDMMR studies, integration across qualitative and quantitative components can increase the overall richness and explanatory power of the dataset. Concurrent analysis helps researchers refine sampling and data collection strategies to ensure that new data contribute meaningful insights rather than repeating existing ideas. Adaptive sampling and iterative interview guide refinement therefore help ensure that data collection is purposeful, efficient, and sufficiently rich to support the generation of meta-inferences.

Failure to refine sampling and data collection strategies based on insights gleaned as the study progresses can lead to wasted time and thinner data, as participants with similar views rehash ideas at the expense of new perspectives, and data collection tools persist with unproductive lines of questioning while missing out on critical emergent concepts. When guided by rich and integrated concurrent analyses, strategies like theoretical sampling can help researchers ensure that they are recruiting participants whose contributions flesh out unforeseen gaps in their data; refining data collection tools such as interview guides ensures that researchers are generating the richest data possible with their participants. We suggest that future work should further develop mixed methods-specific guidance for applying concepts such as information power, particularly to clarify how integration contributes to determining when enough data has been collected.

Conclusion, Limitations, and Future Inquiry

In conclusion, our reflections underscore the importance of making space for qualitative ways of knowing to guide integration within convergent mixed methods designs. By revisiting our initial approach through a qualitatively-driven lens, we identified design adaptations that could have deepened integration and enhanced the utility of our findings. We acknowledge that our reflections are context-dependent, but we hope others can see connections to their own complex problems and contexts. We hope this account empowers other emerging researchers to embrace adaptive, reflective practices that center qualitative inquiry and advance the potential of mixed methods research.