Abstract
Mixed methods (MM) research has gained wide global and disciplinary acceptance. However, MM designs that prioritize qualitative perspectives are not easily recognizable yet offer great potential for researchers. By situating the current state of qualitatively oriented mixed methods (QOMM) research and offering practical guidance, we aim to help researchers leverage design possibilities. We begin by positioning ourselves and describing some distinguishing characteristics to help researchers recognize QOMM designs. We then introduce the key features of a QOMM study and weave illustrative examples into the descriptions of six interconnected design spokes to help researchers navigate a nonlinear design process. Finally, we discuss three useful lessons we learned from our own research experiences and consider their implications to help researchers design future QOMM studies.
Keywords
Introduction
Mixed methods (MM) research has gained wide global and disciplinary acceptance for generating insights not accessible by qualitative or quantitative research alone (e.g., Bergman, 2011; Creswell & Plano Clark, 2018; Johnson & Onwuegbuzie, 2004; Plano Clark, 2010; Poth, 2018; Poth et al., 2022; Shannon-Baker, 2022). Qualitatively oriented mixed methods (QOMM) designs have long been known to offer great potential for researchers seeking to prioritize qualitative perspectives (Cameron, 2016; Morse & Cheek, 2014; Hesse-Biber, 2010; Onwuegbuzie & Poth, 2015). We argue that QOMM has yet to fulfill its potential as a distinct research approach and true recognition for its prevalence in the field. Researcher accounts as well as our own experiences highlight that although QOMM studies are occurring, examples are not easily recognizable within published literature (Archibald et al., 2015; Hesse-Biber, 2018; Morgan & Hoffman, 2021). The lack of illustrative QOMM studies for researchers seeking guidance may be discouraging its use and thus also attributing to its low reported prevalence. In this ‘state of the methods paper’, we seek to help researchers leverage QOMM possibilities by offering practice guidance for advancing its use, usefulness, and uniqueness.
A recent review of literature attributes a lack of visibility, perceived benefit, and power dynamics to the low prevalence of MM designs that prioritize qualitative perspectives compared to other types of MM designs (Morgan & Hoffman, 2021). The authors call for concerted efforts to recognize QOMM (i.e., increased visibility of published examples), to build awareness of the distinct value-added of QOMM (i.e., unique benefits of supplementing qualitative research with a quantitative component), and to create opportunities for showcasing QOMM (i.e., earmarked advocacy for power positioning at conferences, publications, and with funding agencies). Together with our own experiences, we conceptualize visibility, benefits, and power as mutually reinforcing in Figure 1 and advance the pressing need for actionable practices to break the cycle, to bring about change, and advance the current state of QOMM research. To that end, we use the three dynamics as the conceptual and practical frame for this paper: (1) To enhance the visibility of QOMM, we describe a unique niche for QOMM that is distinct from other qualitative and MM approaches; (2) to make explicit some of the benefits of QOMM, we detail examples illustrating the navigation of six interdependent design spokes providing options for researchers from which to pursue their emergent and nonlinear QOMM design processes; and (3) to leverage the power of QOMM research experiences for others to learn from, we describe three practices to represent the lessons we gleaned from reflecting upon our more than two decades of QOMM research experience. The conceptual and practical frame for this paper: Leveraging the visibility, benefits, and power of QOMM.
To begin, we position ourselves, our backgrounds, and our collaborative approach to account for some of the contributors to the insights described in this paper. Then, to help researchers locate QOMM study examples and pursue their own designs, we orient researchers to the distinguishing characteristics of and the types of research that warrant the use of a QOMM approach. Then, to help researchers navigate their emergent and nonlinear design processes and highlight the benefits of this approach, we introduce and weave QOMM study examples into the descriptions of six interdependent design spokes. Finally, to help researchers design QOMM studies and share their experiences, we describe three lessons we have learned and consider their implications. We conclude by proposing future directions and inviting elaborations of the ideas present in this paper.
Positioning Our Collaborative Approach and Our Backgrounds
In this collaboration, we draw upon both commonalities and differences in our research and disciplinary expertise to generate outcomes that would not have otherwise been accessible by either of us working independently. Collectively we bring more than three decades of qualitative, quantitative, and mixed methods research experience as well as extensive backgrounds as instructors, reviewers, and graduate supervisors. Central to the impetus for this work are the lessons that emerged from our joint reflection upon our own experiences, literature searches, example discussions, and their potential usefulness for others to learn from.
Poth is an educator who seeks to understand how to improve learning environments through evidence-based decision making. This work has been undertaken in K–12 and higher education classrooms, social service and business organizations, and a variety of clinical settings. Poth’s worldview of what they value as evidence has been greatly expanded since their initial training in quantitative research as a biologist. As a classroom teacher, they began to question the limitations of the numeric test scores for assessing, reporting, and supporting further learning. An introduction to qualitative and mixed methods research in graduate school was key to locating and connecting with a global community of researchers where they continue learning and honing their craft as a researcher. Poth identifies as a pragmatist in their choice of research approaches and is aware of their own preferences as they strive to generate insights in ways that are appropriate to the research conditions.
Shannon-Baker is a teacher and researcher of educational research methods. They teach courses on qualitative, action research, and MM research as well as courses on educational research and writing research proposals. Shannon-Baker’s approach to research is influenced by their training in creative writing, feminist qualitative inquiry, art history and arts-based methods, and mixed methods research. They come to this work from educational foundations, which is a field that examines cultural differences, influences of power and privilege such as in the form of racism and heteronormativity, and how to promote cultural sustainability in schooling. Philosophically, much of Shannon-Baker’s work aligns with critical, feminist, and poststructuralist paradigms. They use these foundations to teach about and use research methods that aim to support cultural sustainability. In their research, Shannon-Baker tries to center reflexivity, relationship building, voice, and the positive use of subjectivity to influence the research process.
A Distinct Niche for Qualitatively Oriented Mixed Methods Research
With the aim of increasing the overall visibility of QOMM we orient researchers to the distinguishing characteristics of and the types of research that warrant its use. We offer key characteristics in the place of a definition to avoid constraining the possibilities for QOMM and build upon the work of Hesse-Biber (2010; 2018) and Morse and Cheek (2014; 2015) in particular. We intend for our key characteristics and types of research to initiate a conversation that we hope others will extend. Our aim is to help researchers locate illustrative examples, pursue their own designs, and so identify their QOMM studies. To orient this work, we use Hesse-Biber’s (2018) astute yet lengthy definition: An important dimension of any qualitatively driven mixed methods project is a commitment to privileging a qualitative approach over a quantitative approach in that the quantitative component (quan) takes a secondary role by prioritizing qualitatively driven epistemology and methodology as the core of the overall. (p. 6)
We also draw upon the core requirements of QOMM described by Morse and Cheek (2014) who refer to the need for the design to be qualitatively driven and for this to be evidenced in the study purpose, research questions and inductive process. We also acknowledge that QOMM lacks a universal definition or even terminology as the literature is replete with various terms such as qualitative-driven, -prioritized, -emphasized, -oriented mixed methods and our own terminology preferences.
We are often asked when is it appropriate to use QOMM research? In Figure 2, we depict some of the distinctive as well as the common aspects of QOMM and MM research to begin identifying the unique niche for QOMM. Each row completes the sentence, “researchers use this methodology to…” We conduct QOMM research because a problem or issue needs to be explored through the integration of qualitative and quantitative perspectives with the former prioritized. The reasons underpinning the need for studies that prioritize the integration of qualitative information with the quantitative information may be broad—such as those that are represented in the bottom left three rows of Figure 2. Distinct and shared uses of qualitatively oriented mixed methods and mixed methods research.
We conduct QOMM when there is a need for the design to be more qualitatively oriented in what and how information is collected, analyzed, and integrated and in the needs of the research settings, stakeholders, and researchers’ educational background and theoretical underpinnings. For example, Love and colleagues (2018) make the case for their need to emphasize stories collected through in-depth interviews and focus groups with their final online survey offering a supplementary perspective within their three-phase MM study involving 183 multi-generational Black women. The researchers make explicit that “the research primarily relied on the qualitative findings to substantiate the use of the tri-theory model” (p. 480). Further evidence of the prioritizing of the qualitative perspectives is that the findings were organized around the qualitative themes.
We also use a QOMM approach when there is a need to emphasize inductive analytic practices and the insights derived from them. For example, Barnhardt and colleagues (2018) apply a transformative QOMM approach to explore the multidimensional dynamics of contention that are present as campus administrators navigate the process of serving a group of students who are marginalized due to their immigration status. The researchers identify the qualitative data as receiving “interpretive priority by virtue of its capacity to showcase collective meaning, contextual complexity, and temporal depth” (p. 421).
Finally, centering participant voices and perspectives helps researchers to corroborate various data sources to maintain the priority on the qualitative perspectives. For example, Cooper and Hall’s (2016) article describes how a QOMM approach was employed to acquire better understandings of Black male student athletes’ experiences at a historically Black college/university in the southeastern United States. The authors describe their integration strategy to prioritize the qualitative methods saying, “Quantitative findings served as complementary data to corroborate the emergent qualitative themes” (p. 46). It should be noted that none of these authors specifically labeled their design as QOMM, but they had all identified as conducting MM research. The qualitative priority was named or hinted to elsewhere in their manuscripts.
The top five rows of Figure 2 represent the common rationales across both QOMM and MM research. Researchers might use either to address conditions that necessitate multiple types of both qualitative and quantitative data or to position the perspectives of people who have been marginalized to speak back to dominant narratives. QOMM (e.g., Barnhardt et al., 2018) and MM (e.g., Shuayb, 2014) studies of the latter combine the integration of qualitative and quantitative research with social justice initiatives. Researchers might explore alternative perspectives and use sequential designs to use the second phase to explore or explain initial findings in either QOMM (e.g., Fisher et al., 2021a) or MM (Pincus et al., 2006) studies. Researchers also use both approaches to develop innovative quantitative and qualitative data collection instruments and protocols in QOMM (e.g., Love et al., 2018) and MM (Enosh et al., 2015) studies. Together we see that although both QOMM and MM share the common characteristic for the requisite integration of qualitative and quantitative research, the distinctive aspects of and uses for QOMM can help researchers to locate those examples. What remains to be further articulated are the ways that researchers design and use a QOMM approach in their studies.
Navigate Emergent and Nonlinear Qualitatively Oriented Mixed Methods Design Processes
It is sometimes difficult to navigate the various design influences in QOMM and decide how to begin. For some studies, where researchers have already identified the need to prioritize qualitative perspectives in the integration of quantitative perspectives, they might still be unsure about what populations would be most suitable. In other studies, where researchers have already identified the data collection methods to generate the requisite qualitative and quantitative perspectives, they might still be unsure about suitable research settings. Among the key benefits of QOMM is the assumption of an emergent and evolving design process. This may not be surprising as emergent design processes feature prominently as a key characteristic of qualitative research (Creswell & Poth, 2018; Hatch, 2002; Marshall & Rossman, 2015; Ravitch & Mittenfelner Carl, 2016).
An emergent QOMM design process assumes that researchers must be prepared for changes or shifts to initial plans and procedures. What may be less well articulated is that emergent designs also involve nonlinear processes whereby QOMM study features, such as research participants, researchers, and their environments, influence and are influenced by other QOMM study features, such as the rationale for using QOMM, data collection, analysis, and integration procedures. The outcomes of these influences are impossible to predict with any accuracy, yet patterns can often be detected retrospectively. For example, researchers can often gain insights into how contextual influences such as a change in community gatekeeper led to shifts in the data procedures. The ability to adapt to dynamic study circumstances can be a real benefit for QOMM researchers as most research occurs under changeable conditions yet practices that help us navigate are nascent.
Now that we have established that QOMM studies are often emergent and nonlinear in nature, we return to the overall purpose of this paper: to provide practical guidance to help researchers navigate the emergent and nonlinear QOMM design processes through a description of six spokes. In this paper, we refer to the study features that are known to researchers as their design access points. We conceptualize the six spokes as common design access points specific to QOMM and that those spokes are interdependent with one another where each decision affects the other design spokes (see Figure 3). Six interdependent design spokes for navigating qualitatively oriented mixed methods research.
Given the emergent and nonlinear nature of QOMM design processes, it is unlikely or even undesirable for researchers to predetermine all the design spokes (i.e., study features) before they begin. Having multiple design access points provides important options for researchers to pursue their emergent and nonlinear QOMM design processes. To illustrate the practical use of the six spokes and their usefulness in helping researchers navigate their designs, we introduce a QOMM study and weave illustrative examples into the descriptions that follow. We intend for the examples to illustrate the benefits of QOMM and guide researchers in navigating their own emergent and nonlinear design processes.
Introduction of an Illustrative Line of Inquiry for QOMM
Key Features QOMM From Two Publications From the Students with Disabilities in STEM Clubs Line of Inquiry.
In the first publication, Fisher et al. (2021a) investigated the characteristics of STEM clubs in schools in rural settings that serve students with disabilities by using a sequential explanatory MM design (quant→QUAL). They surveyed and interviewed teachers who lead the clubs about the nature of the clubs, their recruitment efforts specifically for students with disabilities, and their training to work with these students in informal learning environments like after-school activities. In the second publication, Fisher et al. (2021b) used a QOMM design to explore the experiences of students with disabilities in STEM after-school clubs according to the students and their parents. The researchers used a sequential explanatory design (quant→QUAL) to explore the complexity of experiences among students with disabilities. The qualitative priority in this study aimed to “center” students’ and parents’ voices “and add richness to the overall study” (p. 3). Fisher and colleagues used an initial online survey phase to identify participants for interviews and focus groups.
Researcher Orientation Spoke
The researcher orientation spoke recognizes the influence of a researcher’s own experiences, education, skills, and knowledge on the research process. As a social process, QOMM involves interactions among people playing different roles in the research (e.g., leading, participating, advising) and people are shaped by the various contexts (e.g., familial, educational, disciplinary) in which they live and work. Elucidating information about disciplinary and methodological backgrounds of those involved in the research, their relationship to the research topic, and with each other, can provide important guidance for others to do the same. Although Poth (2018) makes the case for the value of this information for researchers, she admits that it remains more likely to find it in dissertations than in published articles. We argue that the researcher orientation spoke provides an important design access point for those who are qualitatively inclined and are seeking to pursue a QOMM study.
Although our example study starts from this spoke, not all studies will. Our study demonstrates this access point because it was based on the lead author’s previous research about students with disabilities in STEM clubs. Fisher identified the line of inquiry based on her previous teaching and research experiences. She then sought out other research team members at our institution to join the project based on our complementary strengths. Shannon-Baker, as a mixed methods researcher and methodologist, brought skills and a focus on research design and methodology. Greer brought experience working in informal STEM learning environments with youth and adults. Other researchers joined for different projects based on their experiences working in special education or with students with disabilities, experience with after-school programs for students with disabilities, and/or because they were studying to become a teacher.
Research Setting Spoke
The multiple contexts for a research study greatly influence its design and methods. Plano Clark and Ivankova (2016) frame these contexts in an ecological framework of personal contexts (the researcher’s philosophical and theoretical foundations, background knowledge, education, and skills), interpersonal contexts (the stakeholders in the research such as participants, governing or evaluating bodies, and co-researchers), and social contexts (expectations and norms within the institution, field, region, and country). Where we conduct a QOMM study is important because accessing perspectives qualitatively often depends upon the community relationships that we either establish or build upon. Within qualitative research, an often cited limitation is the extent to which our community and its participants are willing and able to share their perspectives (Creswell & Poth, 2018). Having established access to a community and its participants can provide an important starting place for QOMM research which in turn can have important implications for the credibility of our research findings.
In Fisher et al. (2021a), once the researchers identified their orientations (see the researcher orientation spoke above), they discussed the influence of the research setting. The research team was located in a rural setting surrounded by rural, suburban, and urban emerging school districts. The researchers designed this study to be more qualitatively oriented and exploratory due to the limited research on STEM clubs in rural areas that work with students with disabilities. Thus, their research setting aligned in part with their rationale for using a QOMM approach.
The team was able to work with neighboring school districts through school administrators that the research team had relationships with or that were otherwise connected to our institution. Several of the research team members also teach current practicing teachers or were themselves teachers from these districts. One team member also had connections to district administration in a nearby neighborhood. With these location-specific connections, the researchers were able to emphasize this rural context when positioning the research and how they addressed the unique features of the STEM clubs and teacher training.
Research Participants Spoke
The research participants spoke recognizes the needs and uniqueness of participant groups involved in a QOMM study. Who the participants are or could be, their potential involvement in the research, and researchers’ relationships with them all influence the design of a QOMM study. Specifically, who we seek as our research participants in QOMM has important implications for how we design the sequencing of the qualitative and quantitative components including decisions about sampling, data collection, and analysis strategies. We also need to acknowledge that our procedural data decisions have important implications for how we balance our ethical responsibilities when working with our QOMM study participants. When considering our ethical responsibilities towards participants, we advance the need to specifically think about the possibility of the burdens placed on participants (e.g., time, physical, emotional) and especially those that might be vulnerable in different ways. Such ethical issues have been highlighted as potentially intensified by the collection of both qualitative and quantitative data in MM studies and we would argue requires even further attention in QOMM by both individual researchers and teams (Poth et al., in press).
Through identifying the initial research participants for the example QOMM study, the research team assessed the risks and benefits for teachers and parents. For the first study, the research team initially worked with teachers who had led STEM clubs (Fisher et al., 2021a) to recognize the important organizing work of teachers in under-funded rural settings. As the team sought to expand the perspectives included in their line of inquiry, they designed a second QOMM study with students with disabilities and their parents as participants (Fisher et al., 2021b). This study was important because students with disabilities (and their parents) are best able to share their experiences working with teachers who lead clubs, connecting with peers during activities, and navigating the logistical needs of ensuring their full participation.
The second study detailed the unique factors that students with disabilities were drawn to in joining and staying in clubs (Fisher et al., 2021b). The team documented the importance that parents placed on STEM clubs to give their students and themselves a “semblance of normal[cy]” (p. 5). Parents further described feeling “overworked” trying to handle most of the transportation logistics on their own (p. 6). The details both students and parents shared were made possible by the MM study’s qualitative orientation. In future studies, the team could emphasize their retelling of experiences taking students to competitions, how their interest in a specific STEM field developed, and the powerful impact of encouragement from peers and teachers.
Data Sources Spoke
The data sources spoke highlights that the design of a QOMM study is influenced by the data the researcher has or can get access to (e.g., Brodin & Peterson, 2019). Researchers who start with this spoke as an access point might be either in practitioner or professional settings where they have access to data or are using open access qualitative data. The use of secondary data is increasing in MM yet understandings of and practical guidance for mitigating common pitfalls such as sampling remain scarce (Watkins, 2022). This is not surprising because the limitations inherent in the use of secondary qualitative data are not yet well known (cf. Poth, 2019). QOMM researchers would be well positioned to explore more efficient ways to collect, manage, and analyze their quantitative and qualitative data (Watkins & Gioia, 2015). In addition, we argue QOMM researchers need access to the information about how, when, and why the data was initially collected.
In the studies on the experiences of students with disabilities in STEM clubs, the team collected primary data and did not rely on secondary data. Once the researchers identified potential participants, they selected approaches to data collection that would best help them explore the participants' perspectives. In both studies (Fisher et al., 2021a, 2021b), the team used quantitative data collected through online surveys to select participants for the qualitative interviews and/or focus groups. It is important to note that they initially planned to conduct observations of clubs to shadow students for the second study. However, due to schools moving online and clubs essentially being canceled during the height of the COVID-19 pandemic, the team had to re-envision the second study. They opted instead for online interviews and focus groups to meet participants where they were. This change in the research setting and possible data also influenced the potential participant pool. Whereas the plan for the second study was to work with the same districts from the first study (Fisher et al., 2021a), moving online allowed the researchers to open the study to students and their parents from all around the USA (Fisher et al., 2021b). This expanded the rural focus from the first study to a broader focus on the general supports and barriers students and their families face when attempting to stay in an after-school STEM club.
Prioritized Integration Spoke
Researchers may start from the prioritized integration spoke when they find themselves asking research questions or have a research focus that necessitates a qualitative orientation within a MM design. The phenomena or experience they want to explore is so complex and highly nuanced that a single methodology alone would not be able to explore it fully enough. Brodin and Peterson (2019), for example, used their integration of quantitative mapping data to identify participants for their qualitative interviews in their study of the experiences of CEOs and owners of home care businesses in Sweden across gender, ethnicity, and professional experiences.
In the illustrative QOMM study, the team used quantitative data to identify participants for the qualitative phase in both studies on students’ experiences in STEM clubs (Fisher et al., 2021a, 2021b). The researcher orientations as a team also recognized the need to collect multiple forms of data to address the complexity and nuances of these experiences. Additionally, they chose a QOMM approach because at the core of the inquiry and their research values is the commitment to center the voices and perspectives of participants. The prioritized integration of qualitative perspectives within a MM study was a key design feature.
Theoretical Framework Spoke
A researcher’s theoretical framework might be the access point for their designing of a QOMM study if they are particularly theoretically grounded or want to emphasize a particular theoretical framing throughout the design process. As a design access point, a theoretical framework can help researchers specify the constructs inherent in their research questions (Alavi et al., 2018). The authors argue a lack of clarity about constructs of interest, and their theoretical orientation, can result in the design of flawed studies. We argue, specific to QOMM, that theoretical frameworks can help researchers address the constructs in their research questions and can provide important validation evidence. For example, Love et al. (2018) emphasized their tri-theory model when choosing a qualitative orientation to their mixed methods study on how Black women in Denver, Colorado, USA, who are “committed to social change” (p. 475) describe race across three generations: Millennial, Gen-X, and Baby Boomers. Their theoretical model combined standpoint, intersectionality, and social identity theories. Brodin and Peterson (2019) similarly pointed to their use of intersectionality as a theoretical framework to explain their qualitative priority.
In the illustrative QOMM study, the team did not have a specific theoretical framework from the outset of the study on the experiences of students with disabilities in STEM clubs (Fisher et al., 2021a, 2021b). They instead used conceptual frameworks from the literature reviews to contextualize the research in what is known about the impact of participating in after-school clubs and career aspirations among youth, STEM clubs in rural settings, and initiatives to increase participation of people with disabilities in STEM fields in school and their careers. Describing a theoretical framework spoke as one of six design spokes begins to address the calls for placing greater emphasis on theory within QOMM design processes (Hesse-Biber, 2018).
Practices for Advancing the Current State of QOMM
We describe three lessons we have learned from our review of literature, our reflections upon our own experiences, and from conversations with others to help researchers design their future QOMM studies and share their experiences.
Identify QOMM in our Titles, Abstracts, Keywords, and Methods Descriptions
There remains a pressing need to distinguish QOMM studies if they are to achieve greater visibility. We argue an alternative to the conclusion advanced by Morgan and Hoffman (2021): “qualitatively driven studies remain relatively rare within the field of mixed methods research” (p. 731). Instead of being rare in practice, we contend that the qualitatively oriented studies remain hidden within the mixed methods research literature. We call for researchers to identify the QOMM nature of their studies in their titles, abstracts, and keywords and to publish high quality studies that clearly signal their qualitative priority in the methods section. For example, researchers could include a sentence that clearly identifies the qualitative priority when introducing their MM design such as “this study prioritized the qualitative phase…” (Fisher et al., 2021b, p. 2–3). Researchers could use the notational system in their design diagrams and within their narrative methods descriptions where QUAL (uppercase) refers to the prioritized qualitative component and quant (lowercase) refers to the supplemental quantitative component (Morse, 2017; Morse & Cheek, 2014). There are many ways to identify QOMM in our published reports and that only once we enhance the visibility of QOMM research within the literature can we bring about the recognition it deserves. Including the QOMM label or related phrasing may not be possible in all four locations we have suggested. Instead, researchers should include this specific labeling or phrasing in any of these locations based on what is possible in their publishing outlets.
Rationalize the Value Added for QOMM in our Research Proposals and Reports
To further engage qualitative researchers in considering the possibilities of QOMM, we must be more explicit about its unique benefits in empirical accounts and methodological contributions. We call for rationales for the use of QOMM to specify beyond the simplistic claims of ‘richer’ or ‘deeper’ meta-inferences — instead, we can be explicit about the need to; for example, center on participant voices and experiences for the particular phenomenon (Fisher et al., 2021a). Furthermore, as Bergman (2011) suggests, we could make explicit our use of QOMM to address some of the inherent limitations of qualitative perspectives alone. For example, if we wanted to make the case for, as Hesse-Biber (2010) suggests the relevance of participant experiences in a larger population, a quantitative component could help us make this case while maintaining the focus on the experiences documented qualitatively. Another exciting possibility for QOMM described by Morse and Cheek (2014) involves qualitative researchers integrating their thick descriptions involved in theory building that is complemented with statistical findings. There are infinite possibilities for the use and usefulness of QOMM and being more explicit and specific about the value-added benefits of QOMM might attract some qualitative and mixed methods researchers to explore its potential.
Portray Emergent and Nonlinear QOMM Design Processes
To help QOMM researchers navigate their emergent and nonlinear design processes, we offer the descriptions of six spokes and the illustrative examples to provide practical guidance. Providing QOMM researchers options beyond the selection of a research design has been called for by Hesse-Biber (2018). The recent global COVID-19 pandemic has highlighted that even the best laid studies plans can be disrupted. We must assume that emergence and nonlinearity in our research designs is to be expected — we see this in all research, but it tends to be more talked about in qualitative research. The discussions about complex conditions in MM research have elevated the need to consider procedural adaptations to the field as well (Poth, 2018). Such changes can have important resource implications in terms of the expertise, time, and financial resources to complete a study. We call for learning opportunities (e.g., workshops, courses, seminars) that are not only QOMM-specific but that also equip researchers for the rapidly changeable research conditions they are likely to encounter in practice. We call for the further application of QOMM and the generation of accounts that describe the nonlinear and dynamic design process.
Our discussion of the line of inquiry about students with disabilities in STEM clubs earlier in the manuscript serves to illustrate how researchers can document an emergent and nonlinear design process. Figure 4 visually depicts the navigation among the six design spokes for the illustrative example. The design initiated (as previously described) from Fisher’s ideas and researcher orientation leading to studying the experiences of students with disabilities in STEM clubs (labelled “A”). As the research team came together, the line of inquiry was influenced their shared desire to center participant voices throughout, which necessitated a QOMM design approach (labelled “B”, arrow shown with dashes to identify a general influence rather than a sequential flow). Together in consideration of the rural STEM clubs as the research setting (labelled “C”) and the teacher participants with whom had established relationship (labelled “D”), they designed the first study to involve face-to-face data collection (Fisher et al., 2021a). The data source decisions (labelled “D”) for study 2 (Fisher et al., 2021b) were similarly designed and necessitated adapting because of COVID-19 safety measures, which changed the research context (labelled “F”). As a result, the team had to reconceptualize data procedures (labelled “G”). Additionally, although the studies were not originally theoretically driven, conceptual frameworks and relevant literature were used to discuss the integrated findings and meta-inferences (labelled “H”) in both manuscripts (Fisher et al., 2021a, 2021b). There are many possibilities for how emergent and nonlinear design processes might unfold. The six design spokes provide an important framework to track the points of entry and how research design decisions are navigated as contexts and procedures change. The dynamic process for designing a QOMM study based on our example.
Methodological transparency in MM research has been a much-discussed quality indicator which involves providing detailed accounts of design and procedural decisions (e.g., Levitt et al., 2018), yet few articles offer detailed descriptions about how and why initial plans evolved during the research. We highlight the usefulness of procedural diagrams such as Figure 4 and the reflexivity involved by those involved in the research as useful practices for authentically represent the nonlinear essence of QOMM research. Reflexivity can be described as a researcher’s continuous examination of and reflections upon their explanations of what decisions were made, why those decisions were made, and what influences exist that affected those decisions (Jootun et al., 2009). In so doing, researchers can leverage the symbiotic methodological transparency-reflexive practice relationship that Hiles (2008) describes: “Transparency and reflexivity, therefore, go hand in hand since without transparency, reflexivity is undermined; at the same time, reflexivity obviously promotes transparency” (p. 892). This relationship and increased reflexivity will thereby improve the quality of published work (e.g., Evans et al., 2011; Hesse-Biber, 2010; Popa & Guillermin, 2017). We applaud the efforts of researchers (e.g., Leal et al., 2018) who are not concealing the messiness and challenging current criteria of what constitutes valid evidence in MM evaluation research. We propose the six design spokes as helpful for helping QOMM researchers in their realtime navigation and retrospective tracking of their emergent and nonlinear design processes.
Conclusions
MM designs that prioritize qualitative perspectives afford infinite possibilities yet remain an undervalued opportunity for researchers. We agree that qualitatively oriented mixed methods research studies have received too little attention over the years (Hesse-Biber, 2015; Morse & Cheek, 2014). We call for further texts, special issues, and sessions focused on discussing QOMM studies. The efforts described by Morse and Cheek (2015) are to be applauded that include transparency around review criteria and editorial decisions. These researchers also note the need for clarity in terms and opportunities to showcase in special issues, “if we are to achieve our goal of adding qualitative inquiry to the center of mixed-method and multiple method debates, we need to do more than simply create a new section… a call for a special issue specifically focused on qualitatively-driven mixed method and the contribution of qualitative inquiry to quantitative research. (p. 731). To increase the number of QOMM studies, we advocate for peer reviewers with QOMM-specific expertise for publications as well as conference and funding proposals. Such expertise would enhance the likelihood of equitable treatment and higher quality QOMM. For those willing to advocate for creating opportunities for QOMM studies to be showcased at conferences, in publications, and with funding agencies. QOMM is an innovative and accessible research approach for those seeking to prioritize qualitative perspectives within their MM designs.
As the fields of MM, qualitative, and quantitative research welcomes their next generations, we have hope that attention specific to QOMM can yield the attention it deserves from the global research community. Effectively conveyed within research proposals and reports, researchers can make explicit the specific use, usefulness, and uniqueness of QOMM designs. Specifically, locating QOMM examples is helped by researchers’ identification in title, abstract, keywords and methodological descriptions. Deciding when to use QOMM is helped by researchers’ making explicit the value-added of quantitative perspectives to a core qualitative component. Enhancing its positioning is helped by researchers’ accounts of their emergent and nonlinear QOMM design processes across a variety of avenues. For those willing to embrace an emergent and nonlinear QOMM design process, the six design spokes can help researchers navigate and describe their initial decisions and ongoing adaptations. Together, our discussions of key characteristics, design spokes, and lessons learned about QOMM, we help researchers leverage QOMM possibilities by enhance its visibility, its unique benefits, and power positioning. This is not to say that other practices are not important (i.e., how ethical considerations are intensified by QOMM) but are beyond the scope of this paper. We encourage others to contribute further to this discussion and to the resources to support expanded applications of and possibilities for QOMM.
Footnotes
Acknowledgments
The authors wishes to recognize the contributions of Adrienne Montgomerie to finalizing this work and to thank Linda Liebenberg as the Editor-in-Chief of the International Journal of Qualitative Methodology for the opportunity to contribute this State of the Methods paper.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
