Abstract
Although mixed methods research becomes increasingly popular, the issue that to what extent qualitative and quantitative research methods can be combined is insufficiently explored. To fill this gap, this critical literature review revisits the qualitative-quantitative debate between proponents and opponents of mixed methods research, examines the underlying philosophical assumptions held by two sides, and provides a new perspective to evaluate research combining qualitative and quantitative approaches. A brief review of the origin and development of mixed methods research is provided, followed by a presentation of the divergent opinions of proponents and opponents of mixed methods research and an illustration of why grounded theory, classified as a qualitative research method, can work well with quantitative studies. These ideas contribute to the conclusion rendered here: paradigmatic differences cannot be reduced to the simplistic duality of qualitative-quantitative debate, and the use of compatible paradigms is the key to mixing different research methods.
Introduction
Mixed methods research was initially defined as research designs that involved “at least one quantitative method (designed to collect numbers) and one qualitative method (designed to collect words), where neither type of method is inherently linked to any particular inquiry paradigm” (Greene et al., 1989, p. 256). During the 1990s, advocates of mixed methods research argued that this type of research had “evolved to the point where it is a separate methodological orientation with its worldview, vocabulary, and techniques” (Tashakkori & Teddlie, 2003, p. x). They further stated that mixed methods research represented the third research paradigm, which moved “past the paradigm war by offering a logical and practical method and system of philosophy” (Johnson & Onwuegbuzie, 2004, p. 17). After 30 years of development, mixed methods research has been expanded into a distinct methodology that combines qualitative and quantitative research methods in a single study despite their different underlying principles (Creswell & Garrett, 2008; Creswell & Plano Clark, 2017; Johnson & Onwuegbuzie, 2004). Still, challenges remain concerning the paradigmatic incompatibility between qualitative and quantitative approaches and the incompatibility’s impact on the procedure of mixing (Bazeley, 2002; Bryman, 1984, 2006, 2007). With mixed methods research’s wide utility in various sectors of inquiry, however, practitioners have “neglected philosophy as a criterion of research quality” (Coates, 2021, p. 184) and ignored epistemology and ontology “in favor of a brute methodology” (Baškarada & Koronios, 2018, MMR section, para. 3). Therefore, it is imperative to revisit the incompatibility thesis in the realm of mixed methods research and scrutinize when and how to mix qualitative and quantitative research methods in order to improve the research quality and support well-reasoned research practices (Archibald et al., 2015; Bazeley, 2002; Bronstein & Kovacs, 2013). To address this concern, a critical literature review is conducted to retrieve and analyze the published literature on mixed methods research relating to the paradigmatic (in)compatibility between qualitative and quantitative approaches. This review is structured as follows: first described are the systematic review and thematic analysis of the relevant literature, then demonstrated is an overview of the main positions of proponents and opponents of mixed methods research, followed by a discussion about compatible paradigms as the key to mixing qualitative and quantitative research methods, and finally presented is the conclusion that reaffirms the essential role of philosophical assumptions in mixing different research methods.
Method
The procedure of selecting eligible literature is divided into identification, screening, and inclusion in line with the steps of reporting systematic reviews proposed by Page et al. (2021). During the identification phase, the literature search was applied to two electronic databases, the Web of Science Core Collection (the WOS Core Collection) and the Academic Search Complete (the ASC). The author searched within such sections as Title, Abstract, or Keywords and identified potential studies through using a combination of two or more search terms/keywords (“mixed methods,” “mixed-methods,” “qualitative and quantitative,” “epistemology,” “ontology,” “methodology,” and “paradigm”). To ensure a wide coverage of eligible literature, the search covered publications from 1980 up to the time of writing since mixed methods research has become a distinct approach in the 1980s (Archibald et al., 2015; Creswell & Garrett, 2008). Eventually, a total number of 1,558 records were retrieved from the search, consisting of 1,143 articles from the WOS Core Collection and 415 from the ASC. Then 334 duplicates were removed, leaving 1,224 records. During the screening stage, the author controlled for the quality of literature included in this review and set the peer-reviewed publications as the inclusion criterion. As a result, 111 conference proceedings were excluded as gray literature, which had not been published in a traditional format (Adams et al., 2017; Tong et al., 2020). The title of each remaining record (1,113 in total) was scrutinized and 303 articles were excluded since they had nothing to do with mixed methods research (e.g., “Compiled visualization with IPI method for analyzing of liquid-liquid mixing process” and “Exceptional events classification in warehousing based on an integrated clustering method for a dataset with mixed-valued attributes”).
Following the screening, 810 articles’ abstracts were read to ensure eligibility. Surprisingly, only 70 records were deemed relevant for their focus on when and how to mix qualitative and quantitative research methods at the epistemological, ontological, and methodological levels. The majority of peer-reviewed publications were excluded for their primary or full attention to (1) practicing mixed methods research as an approach to conduct empirical studies (e.g., “Adaptation in families of children with Down syndrome: A mixed-methods design”); (2) discussing how to implement mixed methods research in a given research field (e.g., “Applications of mixed-methods methodology in clinical pharmacy research”); and (3) presenting how to design mixed methods research without delving into the paradigmatic incompatibility between qualitative and quantitative research methods (e.g., “Designing mixed methods research by mixing and merging methodologies: A 13-step model”). These inclusion/exclusion criteria were applied to the full-text review during the inclusion phase. A total of 70 full-text articles were read for further evaluation, leaving 33 publications deemed eligible and included. Additionally, a systematic search of the included full texts’ reference lists was conducted to identify studies that may have been missed. Based on the inclusion/exclusion criteria listed above, 11 articles commonly cited across the included full texts were added into this review (see Figure 1).
PRISMA flow diagram of literature selection.
Thematic analysis (Braun & Clarke, 2006) was used for analyzing the contents of the included articles. The author read and re-read these articles to become familiar with their understandings of mixed methods research. Next, the coding was framed by two questions: (1) what were the views held by these articles regarding the mixing of qualitative and quantitative research methods and (2) what was the argumentation that underscored these views. As the coding proceeded, the included articles’ elucidation of mixed methods research converged to 11 categories under six themes (see Table 1). Then the author reviewed the themes with a research assistant who had read the included articles independently. Through discussion, the author and the research assistant negotiated different viewpoints of specific categories and themes and revisited the relevant articles for clarification when necessary. The process was repeated to ensure an agreement on the categories and themes to be eventually reached between these two researchers.
Categories and Themes Generated by the Thematic Analysis.
Arguments for and against Mixed Methods Research
This critical literature review revealed that arguments for and against mixed methods research centered on the (in)compatibility of paradigms associated with qualitative and quantitative research methods. The term, paradigm, was coined by Kuhn (1970) as a model of scientific practice shared and accepted by a community of practitioners. According to Guba and Lincoln (1994), a paradigm represents a worldview that “defines, for its holder, the nature of the ‘world,’ the individual’s place in it, and the range of possible relationships to that world and its parts” (p. 107). Therefore, a paradigm aims to address three fundamental questions, including the ontological question concerning what reality is, the epistemological question regarding the relationship between reality and inquirers, and the methodological question respecting how inquirers know about reality (Cohen, Manion, & Morrison, 2017; Crotty, 1998; Guba & Lincoln, 1994; Scotland, 2012). Within a paradigm, the methodology is seen as the bridge that brings researchers’ “philosophical standpoint (on ontology and epistemology) and method (perspective and tool) together” (Smith & Sparkes, 2016, p. 6).
In the realm of social science research, qualitative and quantitative research are widely believed to be associated with different paradigms. Constructivism is generally seen as the overarching paradigm of qualitative research for its emphasis on multiple views of reality and researchers’ subjectivity-driven inquiries (Bryman, 2012; Guba & Lincoln, 1989, 1994; Lincoln et al., 2000; Patton, 2015; Petty et al., 2012; Smith & Sparkes, 2016; Timans et al., 2019). However, quantitative research is underpinned by (post-)positivism with the assumption that researchers should adopt meticulous measures to maintain objectivity and eliminate any possible biases in order to uncover objective and absolute reality existing out there (Bryman, 2012; Føllesdal, 1979; Hasan, 2016; Nagel, 1989; Smith & Sparkes, 2016; Timans et al., 2019). The paradigmatic differences between qualitative and quantitative research (see Table 2) have led to intense disagreements between their proponents and opponents, and the long-running debate is referred to as the paradigm war (Bryman, 2007; Coates, 2021; Denzin, 2010; Gage, 1989; Glogowska, 2011; Johnson & Onwuegbuzie, 2004; van Griensven et al., 2014). Influenced by the paradigm debate, scholars articulated their different positions on mixed methods research. In this review, these articulations were found to concern three topics, including (1) whether research methods can be separated from their corresponding paradigms, (2) whether pragmatism can be a good reason for mixed methods research, and (3) whether qualitative and quantitative research methods can be combined without any concern.
Paradigmatic Differences Between Qualitative and Quantitative Research.
Can Research Methods be Separated from Their Corresponding Paradigms?
Some proponents of mixed methods research argue that scholars who hold the opposite opinions of combining qualitative and quantitative approaches confuse research methods with the logic of justification, an essential aspect of epistemology (Johnson & Onwuegbuzie, 2004; Onwuegbuzie & Teddlie, 2003). While admitting that researchers hold up to specific ontological and epistemological orientations, these proponents claim that the logic of justification does not “dictate what specific data collection and data analytical methods researchers must use” since research methods are not inherently tied to epistemology (Johnson & Onwuegbuzie, 2004, p. 15). Similarly, the second group of proponents contends that the key components of research methods, like data gathering tools, sample sizes, generalization, operationalizations, data types, and analytical techniques, are not intrinsically linked to either a qualitative or a quantitative paradigm, and many research is essentially a transaction between qualitative interpretations and quantified data (Bazeley, 2016; Bergman, 2008a, 2008b; Hesse-Biber, 2010; Niaz, 2008; Symonds & Gorard, 2010). Given this situation, there is no evidence for direct links between research methods and epistemology (Liu, 2015). Research methods’ dissociation from epistemology is underpinned by an aparadigmatic stance, which “disputes a strong relationship between philosophy and the research process” (Coates, 2021, p. 173). Mixed methods research practitioners taking the aparadigmatic stance reject the rigid incompatibility of paradigms in the Kuhnian sense and view research methods as tools needed to solve real-world problems (Johnson & Onwuegbuzie, 2004; Niaz, 2008; Onwuegbuzie & Teddlie, 2003; Symonds & Gorard, 2010). The aparadigmatic stance leads to the proposal that researchers’ choices of research methods are not determined by pre-existing incompatible paradigms but by research questions (Bazeley, 2016; Greene et al., 1989; Hesse-Biber, 2010; Niaz, 2008; Philip, 1998; Tashakkori & Teddlie, 1998, 2003; Teddlie & Tashakkori, 2006). With more freedom in choosing research methods, researchers can go beyond the current theoretical limits imposed by the vague grouping of established paradigms and make more powerful and fruitful contribution to research in general (Bergman, 2008a; Philip, 1998; Symonds & Gorard, 2010).
However, the idea of separating a research method from its corresponding epistemology is harshly criticized by some of the earliest opponents of mixed methods research, who argue that methodology cannot be separated from paradigms, and incompatible paradigms inevitably lead to the incompatibility of research methods (Guba & Lincoln, 1994; Howe, 1988, 1992). From these opponents’ purist perspectives, qualitative and quantitative research methods rest on mutually exclusive ontological and epistemological assumptions about reality and knowledge creation, and paradigmatic differences are fundamental to the processes, outcomes, and the corresponding claims of inquiries following these two approaches. For these opponents, it is impossible to juxtapose a research method that embraces all versions of reality as equally truthful and meaningful with another research method that holds on to one single reality as truth. Given the incompatibility of paradigms, there is little room for collaboration or cooperation between qualitative and quantitative research methods. With the cessation of the paradigm war and the increasing popularity of mixed methods research, the second group of opponents calls for attention to philosophical positions marginalized by the quantitative research orthodoxy and expresses concerns over the quality of combining qualitative and quantitative approaches and inconsistent findings they may reveal (Baškarada & Koronios, 2018; Bryman, 2006, 2007, 2017; Coates, 2021; Denzin, 2012; Flick, 2017). In this view, multiple-strategy research or mixed methods research is nothing more than triangulation without a clear understanding of the underlying paradigm since the qualitative approach emphasizes “a relatively open-ended approach to the research process” that frequently “produces surprises, changes of direction and new insights” while the quantitative approach is “by no means a mechanical application of neutral tools that results in no new insights” (Bryman, 2006, p. 111).
Can Pragmatism be a Good Reason for Mixed Methods Research?
To justify mixed methods research, a group of scholars has embraced pragmatism as its philosophical foundation, developed by such writers as Charles Sanders Peirce, William James, George Herbert Mead, and John Dewey (Creswell & Garrett, 2008; Greene, 2008; Johnson et al., 2007; Johnson & Onwuegbuzie, 2004; Morgan, 2007; Tashakkori & Teddlie, 2003). For example, Morgan (2007) drew on the core tenets of pragmatism and proposed to “concentrate instead on concepts such as ‘lines of action’ (from William James and George Herbert Mead) and ‘warranted assertions’ (from John Dewey), along with a general emphasis on ‘workability’ (from both James and Dewey)” (p. 66). Greene (2008) claimed that mixed methods research emphasized “the importance of the dynamic interplay between theory and practice or between thinking/knowing and acting/doing” and further argued that this view was “actually a hallmark of Deweyan pragmatism” (p. 8). As a leading philosophical partner for mixed methods research, pragmatism is commonly believed to offer an alternative paradigm that reconciles the incompatibility of qualitative and quantitative paradigms through integrating different perspectives and approaches in an eclectic manner to inform mixed methods inquiries and offer practice-oriented solutions (Creswell & Garrett, 2008; Glogowska, 2011; Johnson et al., 2007; Morgan, 2007; van Griensven et al., 2014). Grounded in pragmatism, mixed methods research is conceptualized in different ways. Rather than getting hung up on the metaphysical paradigm debate, proponents of pragmatism-grounded mixed methods research adopt a research question-centric approach to knowledge and attend to such practical issues as “how to conduct a mixed methods study, what dilemmas might emerge in doing so, or the practicalities of how to begin thinking about combining both quantitative and qualitative data in a study” (Creswell & Garrett, 2008, p. 328). These proponents further assert that mixed methods research can maximize the synergy and strength of qualitative and quantitative research methods while minimizing their potential weakness (Creswell & Garrett, 2008; Glogowska, 2011; Johnson & Onwuegbuzie, 2004; Makrakis & Kostoulas-Makrakis, 2016; Tashakkori & Teddlie, 2003). In this view, mixed methods research is perceived to lie in the middle of a visualized continuum “with qualitative research anchored at one pole and quantitative research anchored at the other” (Johnson & Onwuegbuzie, 2004, p. 15).
Many scholars raise concerns over mixed methods research’s pragmatic grounding illustrated above (Baškarada & Koronios, 2018; Bergman, 2008a, 2008b; Denzin, 2012; Giddings & Grant, 2007). Giddings and Grant (2007) asserted that pragmatism was captured by post-positivism that generated diverse modes of research inquiries marked by insufficient theoretical reflexivity and eventually guided practitioners to search for a single absolute truth. Given pragmatism’s value of getting things done rather than a preoccupation with epistemological integrity, the notion of mixed methods, in its essence, is nothing but a Trojan Horse for post-positivism (Giddings & Grant, 2007). Although embracing the idea of mixing research methods, Bergman (2008a) criticized advocates of pragmatism for taking refuge in a freely and vaguely interpreted philosophical system, which could benefit both mono method designs and mixed methods designs. Denzin (2012) offered a fiercer critique by describing that classic pragmatism developed by Peirce, James, Mead, and Dewey as a doctrine of meaning that focuses on how actors examine, inspect, and reflect upon the consequences of their actions. Since these meanings cannot be given in advance, classic pragmatism is not a methodology and therefore fails to provide a philosophical base for the mixing of qualitative and quantitative research methods, especially considering these two approaches’ paradigmatic differences (Allmark & Machaczek, 2018; Baškarada & Koronios, 2018; Denzin, 2012). Although pragmatist philosophies have been extended from classic pragmatism to neo-pragmatism marked by anti-realism and strong pluralism, the latter is not the version advocated by proponents of mixed methods research (Denzin, 2012). In this context, arguing for pragmatism’s legitimacy of being the philosophical foundation of mixed methods research “may be too much of a compromise” (p. 5).
Can Two Research Methods be Combined without Any Concern?
Looking beyond the irreconcilable paradigmatic differences, several proponents of mixed methods take a dialectical position, arguing that a dialogical engagement with different paradigms can better address research questions and generate new understanding and insights (Greene, 2006, 2008; Greene & Caracelli, 1997; Greene et al., 1989; Kelle, 2006; Lee & Smith, 2012; McVilly et al., 2008; Morse, 1991). Unlike the proponents in the aparadigmatic stance, practitioners in the dialectical stance recognize the importance and value of different traditional paradigms but claim that these paradigms, as historical and social constructs, can be embraced by a distinctive methodology termed mixed methods approach (Greene, 2008). In their influential work on typology schema of mixed methods designs, Greene et al. (1989) noted that qualitative and quantitative research methods could be combined to achieve the following purposes: (1) triangulation, which examines the convergence or corroboration of results from qualitative and quantitative research methods used to study the same phenomenon; (2) complementarity, which looks for enhancement, illustration, and elaboration of results from one method with results from the other (i.e., two approaches study the overlapping but different aspects of a phenomenon); (3) development, that uses results from one method to develop or inform the design of another method; (4) initiation, which detects paradox and contrast between two methods (often leading to the re-framing of research questions); and (5) expansion, which tends to increase the scope and range of a study by combining two approaches. Focusing on triangulation, Morse (1991) distinguished between simultaneous and sequential uses of qualitative and quantitative research methods to propose four mixed methods designs. Such mixing is deemed valuable and beneficial because “quantitative methods can give an overview about the domain under study and can describe its heterogeneity on a macro-level, whereas qualitative methods can be used to gain access to local knowledge of the field in order to develop theoretical concepts and explanations that cover phenomena relevant for the research domain” (Kelle, 2006, p. 309).
Rather than taking a purist stance, some opponents maintain that qualitative and quantitative approaches entail very different philosophical worldviews and therefore can only be combined for complementarity since the findings from one approach can provide additional insights to the other (Hathcoat & Meixner, 2017; Rossman & Wilson, 1985; Sale et al., 2002; Smith, 1983). Other combinations, such as cross-validation or triangulation, are impossible for mixed methods research in that qualitative and quantitative research methods represent two distinct worldviews (Rossman & Wilson, 1985; Sale et al., 2002). For example, qualitative research methods assume that the phenomenon investigated is subject to individuals’ interpretation and construction of the meanings involved (Sale et al., 2002); in contrast, quantitative research methods assume an external referent by which to compare the claim of truth (Smith, 1983). As a result, triangulation is not possible as the findings from qualitative and quantitative approaches do not converge to a single reality (Sale et al., 2002). In addition to the abovementioned paradigmatic incompatibility, a number of scholars expressed their concerns over practices of combining qualitative and quantitative research methods. Typologies of mixed methods research, according to Timans et al. (2019), standardize “methodological framework for combining methods” and this standardization “amounts to enforcing an unnecessary codification of aspects of research practices that should not be formally standardized” (p. 212). Similarly, Bazeley (2016) claimed that all social research methods were inherently mixed, and the real challenge was to “ensure integration of the quantitative and qualitative aspects of the methods used during the analysis and writing processes” (p. 189). However, mixed methods research practitioners, limited by insufficient guidance in extant literature and time and budget constraints, feel struggled to reflect upon how to integrate qualitative and quantitative data at such earlier stages as data collection, data mixing, and data interpretation and only demonstrate the superficial merging of the data in the section of final discussion (Bazeley, 2016; Zhou & Wu, 2020).
Compatible Paradigms as the Key to Mixing Different Research Methods
The preceding debates about the validity and feasibility of mixed methods research reveal a taken-for-granted presumption that qualitative and quantitative research represent distant universes. As such, the central dispute between proponents and opponents of mixed methods research hinges on whether qualitative and quantitative research methods are mutually exclusive since they hold irreconcilable worldviews or incompatible paradigms. Holding on to the significance of paradigms in designing mixed methods research, this review argues that the qualitative-quantitative dichotomy cannot be equated to the duality of constructivism-(post-)positivism, in large part because of the diverse ideologies and paradigms within qualitative research methods. Guba and Lincoln (1994) noted four underlying paradigms within qualitative approaches: positivism, post-positivism, critical theory, and constructivism. In other words, not all qualitative research methods adopt the constructivism paradigm. Some qualitative methods are heavily influenced by the (post-)positivistic tradition and can be particularly powerful when combined with the quantitative approach. Therefore, the argument here is that qualitative and quantitative research methods can be combined but the key to the combination lies in the compatible paradigms. Taking grounded theory as a representative example, this review will discuss why and how it is compatible with the quantitative paradigm as follows.
Diverse Paradigms within Qualitative Research: The Case of Grounded Theory
As a qualitative methodology, grounded theory provides a systematic analysis of a phenomenon to inductively discover a theory that can explain the phenomenon and fits with empirical data (Corbin & Strauss, 2014; Glaser, 1978; Glaser & Strauss, 1967; Liu, 2015, 2017). This systematic methodology was first proposed by Glaser and Strauss (1967) as a response to quantitative researchers’ harsh critique of qualitative studies and evolved into two major variations during the following thirty years. The Glaserian version of grounded theory holds that an objective reality exists out there; researchers must be as objective and neutral as possible and must rely on systematic analyses to explore reality, and that the theory should fit the substantive empirical data (Glaser, 1978; Glaser & Strauss, 1967; Liu, 2015, 2017). Furthermore, Glaser (1978) argued that grounded theory should be able to explain, predict, and interpret what was happening in a specific area of interest. Taken together, the requirements that grounded theory should both “fit” and “work” constitute the notion of relevance, which is defined as a theory’s ability to grasp the core problems and processes of the subject under systematic investigation (Age, 2011; Liu, 2015, 2017). These core problems and processes emphasized by Glaser (1978) have two properties: (1) the theory can continue over time regardless of the continual variation in places; and (2) the theory can be found in different places at different times (as cited in Age, 2011). The emphasis on an objective and absolute reality, along with the ideas that neutral observation is possible and that theory is transcendent, links the Glaserian view to positivism, specifically the beliefs in an objective (and singular) truth derived from the empirical (and entirely neutral) falsification, and the establishment of facts (Popper, 1972). The version later developed by Strauss and Corbin (1998), also emphasizes external objective reality, unbiased data rendering, and theory verification. But their argument that human beings are active agents links the Straussian version to post-positivism (Charmaz, 2014; Liu, 2015, 2017).
Grounded theory has two features: constant comparative analysis and theoretical sampling (Corbin & Strauss, 2014; Glaser & Strauss, 1967). The former characteristic entails an iterative process of concurrent data collection and analysis, which involves the systematic choice and study of several comparison groups (Cho & Lee, 2014; Liu, 2015, 2017). To put it differently, data analysis and data collection occur simultaneously and the analyzed data guides the following data collection (Liu, 2015, 2017). During the data analysis process, an incident is compared and contrasted with other incidents, empirical data with concepts, concepts with categories, and categories with categories in order to reach higher levels of abstraction and advanced conceptualizations (Cho & Lee, 2014; Liu, 2015, 2017). Strauss and Corbin (1998) provided detailed explanations of constant comparative analysis. The first step is open coding, which identifies and names phenomena (Liu, 2015, 2017). Researchers should be as open as possible to capture as many aspects as possible. Following opening coding, axial coding is conducted, a set of procedures whereby data are put back together in new ways after open coding to make new connections among categories (Corbin & Strauss, 2014; Liu, 2015, 2017; Strauss & Corbin, 1998). A coding paradigm is proposed that involves condition, context, action/interaction strategies, and consequences. The goal of axial coding is to discover concepts or categories through considerable reduction and clustering. The third step is selective coding, in which one category is selected as the core category and other categories are related to it (Liu, 2015, 2017). The goal of selective coding is to generate a storyline along which everything else is draped. When the core categories are developed, the theory can be integrated.
Theoretical sampling is the process of collecting data for comparative analysis (Corbin & Strauss, 2014; Glaser & Strauss, 1967; Liu, 2015, 2017). It involves the recruitment of participants with different experiences so as to explore multiple dimensions of the social process under study (Cho & Lee, 2014; Liu, 2015, 2017). Intertwined with constant comparative analysis, theoretical sampling is conducted in terms of the dimensions of individuals’ experiences rather than their quantity (Corbin & Strauss, 2014; Glaser & Strauss, 1967; Liu, 2015, 2017). During the process of constant comparative analysis, researchers encounter different dimensions and conduct the subsequent data collection following these dimensions (Liu, 2015, 2017). When no new dimensions emerge, the data collection reaches saturation and the sampling can be considered complete. Constant comparative analysis can be used to validate and verify the existing categories, concepts and dimensions. By doing so, researchers can study many more dimensions of a phenomenon. Although Glaser and Strauss diverge on how to conduct grounded theory, they both argue that an objective and singular reality exists and a systematic method should be relied on in order to discover a meaningful theory. The underpinning of (post-)positivism connects grounded theory to the quantitative approach (Johnson & Walsh, 2019).
Combining Grounded Theory with Quantitative Research Methods
Not every qualitative method can be combined with quantitative methods. However, as argued, grounded theory can be combined with quantitative research methods due to the paradigmatic compatibility. The reduction, clustering, and falsification emphasized in grounded theory make it a suitable partner for statistical analysis techniques that emphasize groupings, such as exploratory factor analyses and structural equation modeling.
Mixing with exploratory factor analysis
Exploratory factor analysis is used to detect underlying patterns of a larger number of variables to determine whether information can be condensed into a smaller set of factors or components (Furr, 2011; Haig, 2005). The starting point of exploratory factor analysis is a correlation matrix that displays one variable’s correlation with the rest of the variables. The correlation detected here by exploratory factor analysis is similar to the constant comparative analysis utilized by grounded theory. These two methods aim to build connections among different variables or categories based on their correlation. When the connection among different categories is detected, grounded theory moves to the axial coding to establish the clusters. The reduction is heavily utilized in both processes. Different sub-categories are reduced to a smaller set of categories. For exploratory factor analysis, different variables will be condensed into different factors or components extracted from these variables. Those variables will be loaded into different factors and the clusters come into being. From the correlation matrix to the commonalities matrix and then to the rotated component matrix, variables that are poorly correlated with other variables are removed and analysis will be re-run until the remaining variables are heavily loaded to one factor. The process of exploratory factor analysis is also featured in reduction and clustering, which elevate the extracted factors or components to a higher level. The final factors are similar to the core concepts developed in the selective coding process of grounded theory. Besides the commonalities in terms of reduction and clustering, grounded theory and exploratory factor analysis also involve falsification, which is associated with the possibility of rejecting a null hypothesis. For grounded theory, codes are continuously included and compared with emerging concepts and categories to detect whether new dimensions should be captured (Corbin & Strauss, 2014; Glaser & Strauss, 1967; Strauss & Corbin, 1998). Deduction and verification are heavily utilized by constant comparative analysis, especially in the version proposed by Strauss and Corbin (1998). The theoretical saturation indicates that the theory has covered all possible dimensions. Therefore, this theory is guaranteed to generate a model that closely fits empirical data. In terms of exploratory factor analysis, the significance level will be derived, from which researchers can conclude whether or not extracted factors significantly explain the variance of the chosen variables.
Because of the similarities presented above, grounded theory can be combined with exploratory factor analysis when designing studies. For example, Hsieh et al. (2013) designed a mixed methods study, using findings from the first author’s previous grounded theory research (Hsieh, 2010; Hsieh & Hong, 2010) to generate variables for the subsequent survey. During the qualitative part, the research team grouped 39 medical service providers’ views of medical interpreters into three categories (Patient ally, Health care professionals, and Provider proxy). Constrained by the qualitative interviewing, the research team “refrained from making specific claims about the relationship between the providers’ specialties, expectations, and other variables because such claims are best explained through quantitative analysis” (Hsieh et al., 2013, p. 561). To further examine these relationships, the research team developed an 18-item survey based on previous grounded theory research findings and recruited 293 providers from five specialties to participate in the survey, the second part of the study. Using exploratory factor analysis, the research team identified three components in the survey data that echoed providers’ three views of interpreters mentioned above. They further revealed that significant differences between different specialties were only shown on the component of patient ally. Through triangulating qualitative explorations through a quantitative investigation, the research team extended their prior qualitative research findings and proposed the future research directions to advance the field of bilingual health communication.
Mixing with structural equation modeling
In addition to exploratory factor analysis, structural equation modeling is another suitable quantitative research method that can be combined with grounded theory. As a combination of factor analysis, path analysis, and multiple regression analysis, structural equation modeling is used to construct latent variables and explore their interrelations by a series of observable variables situated in a causal relationship (Acun & Yilmazer, 2019; Anderson & Gerbing, 1988; Salkind, 2010; Weston & Gore, 2006). The pre-condition for researchers to use structural equation modeling is to have an existing conceptual model or theory that is strong and clearly operationalized (Anderson & Gerbing, 1988). Starting from the exiting conceptual model or theory, structural equation modeling defines individual variables theoretically, proposes a measurement model to summarize the interrelationships among these variables, and tests this model via confirmatory factor analysis (Weston & Gore, 2006). Similar to exploratory factor analysis, confirmatory factor analysis in structural equation modeling is characterized by reduction, clustering and falsification since it aims to verify how well the observed variables can be combined to indicate a smaller number of underlying hypothesized constructs or clusters termed latent variables with the highest factor loadings. Then a structural model is put forward to specify the hypothesized structural relationships among latent variables and this step is similar to the axial coding of grounded theory mentioned above. Therefore, structural equation modeling can be used to test how parsimoniously and accurately the associations observed in empirical data fit the structural relationships among variables.
The test-theory perspective, together with the emphasis on reduction, clustering, falsification, structural relationships, and fitting between data and theory, pairs structural equation modeling with grounded theory. Engaging in the constant interplay between data analyses and theory building, practitioners of grounded theory develop different categories, delineate their associations with each other, and arrange them based on these relations to create a conceptual framework that can explain the data and explain the phenomenon under study (Corbin & Strauss, 2014; Glaser & Strauss, 1967). Based on the conceptual framework deriving from grounded theory, structural equation modeling can then be used to detect whether large amounts of newly recruited empirical data fit the current model. Combining grounded theory and structural equation modeling, Acun and Yilmazer (2019) designed a mixed methods research to understand the sound environment’s impact on visitors’ subjective museum experience in a historical museum. Guided by the Straussian version of grounded theory, seven core categories were generated and hierarchically patterned within a conceptual framework that explained how different factors influenced 15 interviewed visitors’ perception of the indoor soundscape of the museum. During the subsequent questionnaire survey, the conceptual framework created in the spirit of grounded theory was used as the basis to carry out structural equation modeling analysis. In addition to the demographic information, the remaining questions were designed based on findings from the qualitative part to ensure their associations with the aforementioned core categories. Through structural equation modeling, the conceptual framework generated by grounded theory was tested against the observed data from 113 questionnaires. The statistic results only supported the relevance of three core categories and the hypothesized structural relationships among them. Based on the refined conceptual framework, Acun and Yilmazer (2019) concluded that visitors’ positive interpretation of the sound environment (the central phenomenon under study as well as the first core category, Interpretation) could create a livable environment with a unique atmosphere (the second core category, Context), and such an atmosphere could increase these visitors’ interaction with the museum and more importantly positively affect their museum experience (the third core category, Outcomes).
Discussion
As noted above, there were 810 articles deemed eligible at the end of the screening stage, and this number decreased sharply to 33 at the beginning of the inclusion phase. Such an extremely high rate (95.9%) of exclusion partially reflects practitioners’ common neglect of philosophical assumptions when designing, implementing and reporting mixed methods research, as the recent literature disclosed (Baškarada & Koronios, 2018; Coates, 2021). The overlooked essential role of philosophical assumptions in mixing qualitative and quantitative research methods leaves room for the aparadigmatic stance, which dissociates the methods from epistemology and regards them as problem-solving tools merely determined by research questions. To justify the research question-centric approach, pragmatism is widely embraced as an alternative paradigm, proposing that “research questions can guide decision making and replace philosophy as the foundation of a study” (Coates, 2021, p. 173). However, pragmatism’s prioritization of research questions makes it a versatile match for both mono method designs and mixed methods research (Bergman, 2008a). The insufficient interpretation of pragmatism and the inadequate critical reflection on its different forms have raised researchers’ concerns over whether this philosophy should be taken as justification for using mixed methods research (Coates, 2021; Denscombe, 2008; Hesse-Biber, 2015).
In addition to those holding the aparadigmatic stance, many proponents of mixed methods research recognize the paradigmatic incompatibility between qualitative and quantitative research methods. As illustrated previously, one group of advocates turn to pragmatism for help, which is believed to reconcile the inter-paradigm incompatibility and maximize the synergy and strength of different research methods. The other group treats mixed methods research as a methodology, which is believed to better address research questions through facilitating different paradigms’ dialogical engagement with each other. Regarding what the synthesis exactly is and how it can be precisely conducted, the theorization presented above still remains ambiguous partially because the assumed inter-paradigm commensurability can be neither convincingly justified nor concretized in empirical studies (Baškarada & Koronios, 2018). Consequently, inconsistency and even contradiction are not uncommon in mixed methods research (Baškarada & Koronios, 2018; Pierre, 2014). Given the complexity of research needs and the lack of exemplars and guidance, practitioners from different research fields express their uncertainties in designing mixed methods research and struggle with maintaining the quality of integration within stringent budgets and time constraints (Corden & Hirst, 2008; Mayoh et al., 2012; Zhou & Wu, 2020). Against this backdrop, this review shares the same viewpoint with some published literature, stressing the absolute necessity of reporting the epistemological and ontological groundings in every mixed methods research to facilitate readers’ understanding and evaluation, reduce biases and guarantee the research quality (Baškarada & Koronios, 2018; Coates, 2021; Frels et al. 2015; Sommer Harrits, 2011).
Conclusion
To conclude, this review scrutinizes arguments for and against mixed methods research revolving around the ontological, epistemological, and methodological disputes along the following three dimensions. Regarding whether research methods can be separated from their corresponding paradigms, proponents of mixed methods research advocate research methods’ independent position while opponents of the mixing insist on qualitative and quantitative approaches’ inherent ties to distinct philosophical assumptions. In terms of whether pragmatism can be a good reason for mixed methods research, proponents embrace this concept as the third paradigm to reconcile the incompatibility of paradigms, but opponents refute this practicality-oriented approach for being essentially post-positivistic, vague, and methodologically unsatisfactory. When coming to whether research methods can be combined without any concern, proponents take a dialectical stance to promote diverse combinations of qualitative and quantitative approaches, whereas opponents believe in the hindrance of incompatible philosophical worldviews to the unconstrained mixing of different research methods and regard integration as the real challenge for mixed methods research.
Based on the critical review, the author argues for the essential role of philosophical assumptions in mixing qualitative and quantitative research methods. As Coates (2021) elucidated, the essential role means that philosophical assumptions should take precedence over research practices and “be communicated when reporting research” (p. 186). Given the essentiality of philosophical underpinnings discussed here, this review advocates mixed methods research but insists on taking the paradigmatic compatibility as the prerequisite for mixing different research methods. After revisiting the long-running paradigm debate, the author opposes the taken-for-granted equation between the qualitative-quantitative dichotomy and the duality of constructivism-(post-)positivism. Then, the author calls for further attention to heterogeneity within qualitative research methods, which represent a large collection of research methods under a convenient, though inadequate, heading (Bergman, 2008b, 2018). To justify the argumentation, the author takes grounded theory as a representative example and illustrates how its grounding in (post-)positivism and emphasis on researchers’ independent stance make this methodology a suitable partner for the quantitative approach when designing mixed methods research (Johnson & Walsh, 2019).
Considering the limited space, the two empirical studies selected for grounded theory’s mixing with quantitative research methods only exemplify an exploratory sequential approach in designing mixed methods research. This design is characterized by an initial implementation of qualitative data collection and analysis and a subsequent quantitative data collection and analysis (Creswell, 2003). Also, the literature beyond the two chosen electronic databases was not scrutinized, and the gray literature that may have yielded insights into mixed methods research was not consulted. Given these limitations, three suggestions are proposed to further advance the theorization of mixed methods research and support well-reasoned empirical research practices in the future. First, future research should explore other possible types of mixing grounded theory with the quantitative approach. Next, future research should broaden the scope of the literature search to include more publications pertinent to mixed methods research. Last but not least, future research should examine more qualitative research methods for the feasibility of partnering with qualitative research methods. For example, qualitative content analysis and behaviorist ethnography are rooted in (post-)positivism (Ahmad, 2007; Crede & Borrego, 2013; Mayring, 2014; Roy & Banerjee, 2012). Hence, these two qualitative research methods may have the potential to pair with specific quantitative research methods in mixed methods research.
Footnotes
Author Note
Yang (Vivian) Liu (PhD, University of Oklahoma, 2017) majoring in intercultural communication, is working as an assistant professor in School of English and International Studies, Beijing Foreign Studies University. Dr. Liu has two research lines. One attends to the qualitative research methods, with the emphasis on how the qualitative approach can be better integrated into mixed research methods. The second focuses on the exploration of sojourners’ and immigrants’ Other-identity, which emerges from intercultural communication. Dr. Liu’s studies were published in both United States and China, and she has experiences both in academia and industries.
Declaration of Conflicting Interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This article is funded by Humanities and Social Science Fund of Ministry of Education of the People’s Republic of China (Grant number: 18YJCZH108) and Beijing Municipal Office of Philosophy and Social Science Planning (Grant number: 18JDXCB002).
