A discussion of some controversies in mixed methods research for emerging researchers

Abstract

Mixed methods research has become an important approach to research worldwide. The combination of qualitative and quantitative research methods has made it possible for a deeper and broader understanding of multifaceted phenomena, thereby offering readers more confidence in research findings and conclusions. The use of mixed method designs became more established in the 1980s and early 1990s, but some controversies surrounding the approach remain. Nonetheless, experts in the field of mixed methods research have continued to work on the central premise that the use of qualitative and quantitative approaches, in combination, provides a better understanding of research problems than either approach alone. This concept paper discusses some of the known controversies around mixed methods with the aim of providing useful insights to emerging researchers interested in learning the methodology.

Keywords

Mixed methods research controversy emerging researchers qualitative and quantitative approaches research paradigms

Introduction

It is well-known that mixed methods research or mixed research is one of the three major methodological approaches used in modern research or investigations (Creswell, 2011; Johnson et al., 2007; Maxwell, 2016). Mixed methods research (MMR) comprises a collection and analysis of both quantitative and qualitative data within a study (Creswell, 2011). Creswell and Plano Clark (2007) defined MMR as:

A research design with philosophical assumptions as well as methods of inquiry. As a methodology, it involves philosophical assumptions that guide the direction of the collection and analysis and the mixture of qualitative and quantitative approaches in many phases of the research process. As a method, it focuses on collecting, analyzing, and mixing both quantitative and qualitative data in a single study or series of studies. Its central premise is that the use of quantitative and qualitative approaches, in combination, provides a better understanding of research problems than either approach alone (Creswell and Plano Clark, 2007; as cited in Creswell, 2011: 3).

The use of mixed methods to study complex social phenomenon goes back to the mid 19th century where most investigators started using both qualitative and quantitative approaches in single studies (Maxwell, 2016). For instance, in 1898, DuBois engaged in field work to obtain data while studying 8000 inhabitants of a slum in Philadelphia, using in-depth house-to-house interviews, a phenomenon which seemed strange at the time (Bulmer et al., 1991; as cited in Jerabek, 2016). The use of mixed methodologies in the social sciences and health services gathered more traction in the mid 20th century, and subsequently developed in the 1980s and early 1990s (Creswell and Plano Clark, 2011; Teddlie and Tashakkori, 2003).

The basic assumptions of qualitative and quantitative research

The qualitative approach to research is often described as a naturalistic, interpretative approach that seeks to inquire into a phenomenon of interest, focusing on a detailed account of participant perspectives (Flick, 2009). Denzin and Lincoln (2011) defined qualitative research as a set of interpretive approach concerned with the transformation of the world through a combination of data collection methods (fieldnotes, interviews, conversations, photographs, recordings, and memos to self), within the natural settings of participants, in an attempt to explain or interpret a phenomenon. The emphasis of qualitative research is on the understanding of the phenomenon being studied rather than simple measurement. Characteristically, qualitative research aims at offering a comprehensive and interpreted understanding of the social world of study respondents; and the application of non-standardized, adaptable approaches to data collection that are complex or sensitive to the social setting of the study and can be revised for each participant or case to allow for the exploration of emergent issues (Guba and Lincoln, 1994; Ritchie et al., 2013). The process of qualitative research is largely inductive since interpretation is grounded in the data, yet all observations are “theory-laden” and are connected by thoughts and assumptions. Qualitative researchers are of the view that “social reality cannot be captured or portrayed “accurately” because there are different perceptions and understandings, although some researchers still aim to “represent participants’ meanings as faithfully as possible” (Ritchie et al., 2013: 12). Conversely, it is worth mentioning the deductive (theory-testing) capacity of qualitative research. For instance, Kirk and Miller (1986) identify good qualitative method as a means of oscillating inductive and deductive techniques to data synthesis within the context of the study. The qualitative deductive approach has the ability to examine or analyze theories through testing and critical evaluation of the outcomes of the theories (Creswell, 2007; Hyde, 2000). This, however, demands an empirically collected data driven by the underlying theory to confirm or reject it (Creswell, 2014; Lincoln and Guba, 1986).

Quantitative research, on the other hand, involves a descriptive, survey-based social inquiry and controlled experimentation. It involves a formal, objective, and systematic process in which numerical data (such as clinical trials or census data) are used to obtain information about the world (Burns and Grove, 2005; Rolfe, 2013). The focus of quantitative research is measurement and analysis of data to establish relationships among variables under investigation. Quantitative researchers assume that research and enquiry should be objective, beginning with a clear hypothesis of a relationship between cause and effect (Hathaway, 1995). This involves developing research instruments and identifying samples with the aim of measuring a phenomenon and making generalizations after testing hypotheses (Creswell, 2003). Quantitative approaches situate research in the postpositivist paradigm (Creswell, 2013), with studies being designed as experiments, quasi experiments and surveys using preplanned data collection instruments to produce statistical data (Bahari, 2012). Research here is deductive, using theories at the beginning of a study to frame research questions or hypothesis (Bryman, 2004; Creswell, 2003). However, it is worthy to note that inductive approaches starting with quantitative data to establish relationships among variables and then moving to qualitative data have contributed immensely to theory development (Jebb et al., 2017; Woo et al., 2016). Although inductive research is directed by the “investigator’s substantive knowledge, tacit predictions are often made about where meaningful patterns will occur” (Jebb et al., 2017: 7). The combination of both deductive and inductive approaches in a study is key in maximizing data use, with the benefits depending on the goals of the broader research project.

Often, researchers concentrate more on the selection of methods for investigations without considering the basic assumptions fundamental to the research approach. The lack of detailed knowledge of the assumptions informing the methodology can affect the validity of research findings and conclusions (Baškarada and Koronios, 2018). Researchers’ knowledge of the philosophical foundations or assumptions help to highlight the strengths and weaknesses of the study approach. The assumptions that follow a specific research approach have the power to shape the researchers’ questions, roles, and even, the methodology of choice (Baškarada and Koronios, 2018; Hathaway, 1995). In order to reduce or avoid the difficulty connected with validity criteria, some proponents have contended that the selection of the study approach (qualitative or quantitative) should not be decided at the method level but at the beginning of the study (Guba and Lincoln, 1981). This assertion indicates that both qualitative and quantitative research methods have their own weaknesses and strengths; hence none of them can be used to address all the problems that characterize human societies around the world.

The two approaches, therefore, require the support of each other, grounded in the basic assumptions underpinning each of the approaches to make the methods more rigorous to guide the study (Cohen et al., 2000; Tuli, 2011). Using the wrong assumptions to select a methodology or choosing a methodology without considering the needed assumptions at every step of the study could bias the outcome. In a bid to solve these epistemic arguments among researchers, the claim that qualitative approach can make up for the limitations in quantitative approach or quantitative approach can redress deficiencies in qualitative approach was coined, which brought MMR. Mixed methods research is an approach to knowledge (theory and practice) that embraces multiple viewpoints, perspectives, and positions to include the stands of qualitative and quantitative research (Johnson et al., 2007).

The value of mixed methods research

Mixed methods research has contributed immensely to academia through its application in the social sciences (Creswell, 2011; Morgan, 2007). The integration of qualitative and quantitative research methods has made it possible for a broader understanding of complex phenomena, thereby offering readers more assurance in research findings and conclusions. The integration process makes it possible to separate mixed methods from other research paradigms (Creswell, 2011; Creswell and Plano Clark, 2007; O’Cathain et al., 2010). Other researchers have argued that the limitations in using one approach gave rise to the use of mixed methods in health services research (O’Cathain et al., 2007), with the view that such “mixing” will strengthen research findings. The use of MMR, therefore, allows investigators to uncover the drawbacks in complex social issues that the qualitative or quantitative method could not identify or address by itself, and to make policy recommendations to engage policymakers to provide permanent solutions.

Mixed methods approach allows researchers to gain a deeper and broader understanding of a given phenomenon compared to studies that use monomethod (Creswell, 2011; Jones et al., 2015; McKim, 2017; Tashakkori and Teddlie, 2003b). This is because of the belief that it helps to explore complex issues in society for a richer understanding of measures and unintentional processes leading to a better dissemination of the generated outcomes (Coyle and Williams, 2000; Papadimitriou et al., 2014; Tashakkori and Teddlie, 2003b).

The value of mixed methods could also be seen from its usefulness as one of the approaches driving specific disciplines in research. According to Fàbregues et al. (2021), while mixed methods and approaches remain relevant in several disciplines, they are very popular in the fields of sociology, education, nursing, and psychology. They contend that a number of articles utilizing mixed method approaches proportionately emanate from professionals in these fields and that several methodological reviews have come from subfields of these disciplines. Professionals in these disciplines have also been particularly instrumental in the growth of mixed methods research as they (the disciplines) are marked by clear boundaries that make it easy for relevant comparative understanding (Ivankova and Kawamura, 2010). Thus, these disciplines are among many others driving mixed methods in academia and research.

Regardless of the significant contribution of MMR over the years in finding solutions to the many complex social and health-related problems, there are still many controversies around the use of MMR. Here, we present a brief vignette of some of the controverses in MMR with a focus on emerging researchers interested in learning the methodology. We aim not only to encourage novice researchers to use this methodology amidst these criticisms, but to also offer some insights in the literature on how to address these controversies in the research process.

The consideration of MMR as a separate or distinct paradigm that leads to an artificial separation of the quantitative and qualitative approaches

As already discussed in this paper, mixed methods stem from a combination of qualitative and quantitative approaches. Researchers generally agree that all research are based on specific paradigms although this might not be clearly stated (Creswell, 2013; McChesney, 2020). The challenge for MMR is the identification of suitable paradigms justified across the mixed methods research process (McChesney and Aldridge, 2019). In general, researchers associate positivism and postpositivism with quantitative methods while interpretivist and other paradigms have been related to the qualitative strand (Creswell, 2013; Johnson et al., 2007). This has led to some researchers taking entrenched positions either for positivism (Maxwell and Delaney, 2004; Schrag, 1992) or interpretivism (Guba and Lincoln, 1989; Schwandt, 2000), hence the so-called paradigmatic wars (Johnson and Onwuegbuzie, 2004).

Freshwater (2007) acknowledges controversies about “opposing paradigms” and “commensurability of epistemologies” (p. 140). This seemingly uncompromising division of paradigms and consequently approaches and methods poses a serious challenge to mixed methods researchers. The question here is: if these paradigms and approaches are completely different and incompatible, then what is the justification for “mixing” them? McChesney and Aldridge (2019) questions the validity of studies that ignore the paradigm issue by simply collecting, analyzing, and integrating quantitative and qualitative data with no regard to the ideological underpinnings of research (see for example Shannon-Baker, 2016). To address this controversy, McChesney (2020) proposes a combination of qualitative and quantitative methods situated in one overarching paradigm. In this way, the issue of incompatibility is addressed as a single philosophical assumption frames the research. Another way of addressing this controversy that has been noted is the integration of multiple paradigms within a mixed methods study. The key here is the researcher being conscious of the nitty-gritty of each philosophical worldview and skillfully adopting an iterative approach of framing a mixed method study by going back and forth within paradigms in order to obtain deeper insights into the phenomenon under study (Greene, 2012). In doing this, there is the tendency to focus more on one paradigm than the other. This is, however, not a problem in MMR since the study purpose will determine whether one paradigm predominates the research process, with the other supporting, explaining or clarifying certain issues. For example, if one was interested in understanding the experiences of people living with a particular condition that has got them stigmatized, it would make sense to collect comprehensive qualitative data. However, to know how prevalent this condition is, it would be valuable to conduct a survey (quantitative), which could help to understand the generalizability of the phenomenon within a broader study whose primary contribution arises from the qualitative work (See Creswell, 2013).

The overriding influence of postpositivist underpinnings in mixed methods research

In general, mixed methods have been situated in the pragmatic paradigm (Creswell, 2013). Although this is a controversy in MMR as some researchers believe that pragmatism could be available in all research ideologies and should therefore not be construed as a paradigm in its own right (Giddings and Grant, 2007), another issue or tension has been noted in the literature. This controversy relates to the dominance of a postpositivist approach to MMR in some disciplines such as health and social sciences. This position has been spearheaded especially by Giddings (2006) and Giddings and Grant (2007) who accentuated that mixed methods were being used as a façade to propagate postpositivist ideologies. They particularly felt that most of the so-called mixed methods studies focused mainly on methods and very little on methodologies, with qualitative aspects virtually non-existent. This was noted earlier by Barbour (1999) who stressed that quantitative aspects in mixed methods usually acted as partial correctives to qualitative parts (see Bazeley, 1999). Freshwater (2007) asks whether the qualitative and quantitative strands could be given equal weight in MMR. However, some researchers allay such fears by stating that the dominance of the postpositivist approach is a consequence of early career researchers who have little understanding in a field that is rapidly developing (Miyata and Kai, 2009; Niglas, 2009). Thus, this is just a case of poor research design and implementation and not necessarily a problem with MMR ideologies. In addressing these issues, De Lisle (2011) suggests that novice researchers should explicitly declare philosophical underpinnings as well as design frameworks when setting up and implementing MMR projects.

Validity issues in mixed methods research designs

Although mixed methodologies have been widely used in research (Hesse-Biber and Johnson, 2013), the issue of validity associated with such designs has been a bone of contention (see Mathison, 1988). This challenge has been acknowledged by mixed methods researchers, with Onwuegbuzie and Johnson (2006) agreeing that due to their nature of blending the unique strengths and “nonoverlapping” shortcomings of qualitative and quantitative methods, it becomes a challenge to assess the validity of MMR. Research design in mixed methods is a product with many primary and secondary characteristics that needs critical attention. The study design should take into consideration the research questions or objectives (Johnson et al., 2007; Schoonenboom and Johnson, 2017; Venkatesh et al., 2016). The design should be concurrent or sequential in order to fully describe the phenomenon under investigation. In a concurrent design, both components (qualitative and quantitative) are executed simultaneously, whereas in a sequential design, the quantitative component precedes the qualitative component, and vice versa (Schoonenboom and Johnson, 2017). Here, a key challenge has been noted in the mixed methods literature, with Guest (2013) and Creswell and Plano Clark (2011) noting that the adoption of a number of terms to denote the types of mixed method designs foments confusion in the nomenclature. They, however, believe that this will improve with time as the field of MMR develops. For validity purposes, however, the design should be transparent to address all issues related to credibility or trustworthiness as illustrated by mixed methods stalwarts (Onwuegbuzie and Johnson, 2006; Tashakkori and Teddlie, 2006). This means that the design should portray the following characteristics as explained by Onwuegbuzie and Johnson (2006) and Tashakkori and Teddlie (2006).

Suitability: The suitability of a study design implies whether the approaches of the study are right to answer the research question(s) and/or the design is in consonance with the research question.

Analytical adequacy: This implies whether the methods for data analysis are sound or appropriate for addressing the research question(s).

Integrative efficacy: This determines whether the meta-inference adequately integrates the inferences stemming from the quantitative and qualitative stages of the study.

Inference transferability: Represents the ability to generalize the results from both quantitative and qualitative research, which involves population transferability (i.e. transferability to other individuals, groups, or entities), ecological transferability (i.e. transferability to other contexts or settings), and operational transferability (i.e. transferability to other approaches of evaluating behaviors).

In this regard, researchers have been advised to be mindful of the research questions and the purpose of the study (Venkatesh et al., 2016).

Again, since mixed methods design involves the collection, analysis, and integration of both quantitative and qualitative data in a single or multi-phased study (Hanson et al., 2005), the data collection in a mixed methods study combines both non-numerical and numerical data. Non numerical data use unstructured questionnaires while the numerical data use structured questionnaires. The integration process can take three forms (Tashakkori and Teddlie, 1998). This includes “concurrent mixed analysis where one analyzes both qualitative and quantitative data simultaneously; sequential qualitative-quantitative data analysis where one analyzes qualitative data then quantitative data; and sequential quantitative-qualitative data analysis where one analyzes quantitative data then qualitative data” (Venkatesh et al., 2016: 12). The argument is that qualitative and quantitative methods, as separate approaches, have distinct techniques of validating research findings. For instance, in a quantitative approach, validity is seen as the extent to which a concept is precisely measured depending on the reliability or the accuracy of the instrument used for the measurement, with reliability here implying the consistency of a measure in the study (Fletcher et al., 2005; Heale and Twycross, 2015). Therefore, the validity and reliability together form the rigor in quantitative research. Rigor in qualitative research is ensured using quality criteria developed by experts in the field of qualitative research. Examples of such criteria include: Tracy’s (2010) Eight “Big Tent” for excellent qualitative research: Worthy topic, sincerity, credibility, resonance, significant contribution, ethics and meaningful coherence (Tracy, 2010: 840).

Worthy topic: Refers to how relevant, timely, and interesting the research topic is within a given environment or discipline. Rich rigor discusses the theoretical construct and scientific techniques to data collection and evaluations of data gathered from the field or participants.

Sincerity: Implies the researcher’s positionality or standpoint and self-reflexivity to curb biases, while ensuring transparency in the approaches employed.

Credibility: This denotes the trustworthiness or confidence of the research results regarding triangulation, thick description of data, and member reflections.

Significant contribution: Significant contribution talks about the additional knowledge derived from the study in relation to theory, practice, morality, and methods.

Ethics: Ethics describes whether the investigator followed due processes to obtain ethical clearance prior to data collection.

Meaningful coherence reflects whether the study attained its intended purposes through methods and procedures that fit the stated goals of the research.

Whittemore et al.’s (2001) contemporary synthesis of validity criteria made up of three levels: primary criteria-integrity, credibility, authenticity, and criticality; secondary criteria-explicitness, vividness, creativity, thoroughness, congruence, and sensitivity; and techniques.

Guba and Lincoln’s (1989) criteria which consist of: credibility; transferability; dependability; confirmability among others.

This discussion regarding design and evaluation raises a few tensions about the validity of mixed methods. For example, is picking one of the core mixed designs tantamount to validity regardless of the way the research is executed? Similarly, when it comes to evaluation criteria when mixing is such a priority in MMR, is it sufficient to evaluate the strands separately? At what point should integration take place in MMR?

Validity (content and internal validity) issues in mixed methods are checked using several elements such as: member checks, triangulation, peer examination, researcher’s bias, and many more. In addition to the aforesaid quality criteria, Onwuegbuzie and Johnson’s (2006) nine types of validity criteria: integration; inside-outside; weakness minimization; sequential; conversion; paradigmatic mixing; commensurability; multiple validities; and political legitimation are often used to ensure rigor in mixed methods research. To Slevitch (2011), the difference between qualitative and quantitative approaches is mainly about the logic of justification, and not methods since each approach emanates from unique ontological and epistemological standpoints which represent two paradigms. Therefore, evaluating MMR on the basis of methods alone might not provide a fertile ground for validity. Wilkinson and Staley (2019) analyzed 698 manuscripts (which were purporting to be of mixed method designs) in the field of literacy research. Of these, they realized some inherent challenges associated with the “mixed method” claims of the authors stemming from underdevelopment of the quantitative or qualitative components in the data collection or analysis, lack of focus, weak integration or articulation of the quantitative and qualitative components, and flawed logic of inquiry (Wilkinson and Staley, 2019). McChesney (2020) adds her voice to this discussion, stating that many “mixed studies” are actually separate qualitative and quantitative studies purporting to address the same research question. She adds that findings from both methods may be reported separately in these so-called mixed studies, with integration virtually non-existent. In other words, most of the challenges related to validity in MMR would be addressed if researchers paid attention to these considerations (proper development of the qualitative and quantitative strands during data collection and analysis and a sound integration process). This also suggests that the way MMR is executed is more important in terms of validity than merely picking one of the mixed methods designs.

In her study on how researchers use the legitimation policy as a means of ensuring validity in mixed method research, Perez (2019) posited that the quality of mixed method research needs to be established at the planning stage of the study, and be focused on the quantitative and qualitative aspects, as well as the whole study. This, in her view, makes the study more valid than focusing on the stage of integration (Perez, 2019). This is supported by Fetters and Molina-Azorin (2017) in their Integration Trilogy where they emphasized that integration of quantitative and qualitative approaches can take place at multiple stages of the research process contrary to the dominant belief that this should happen at the analysis stage. Getting “integration” to permeate the mixed methods research process could be achieved through designing research questions based on both quantitative and qualitative data rather than setting separate questions for each type of data and collecting data such that “at least some participants contribute both qualitative and quantitative data, rather than having separate samples for the two data types” (McChesney, 2020: 102).

Long (2017) highlighted the shortcomings of earlier works which treated the validity in mixed method research by focusing on the quality of research than validity per se, noting that validity was treated as a by-product and not the main product. Accordingly, the use of Habermas’ three validity claims was proposed to validate mixed methods research viz objective, subjective, and normative claims (Long, 2017). Objective claims encompass things which exist in the physical and external world and the associated relationships thereof, and signifies what is and what works (Denis et al., 2014; Long, 2017). On the other hand, subjective claims connote the subjective state of an individual such as the feelings, thoughts, and desires in the person’s “own” world (Long, 2017). As well, the normative claims involve what is supposed to be right and appropriate in the society and what are considered as wrong and unworthy. In proposing these mechanisms to establish the validity in mixed method research, Long (2017) further argued that “addressing validity issues in mixed methods research from a perspective of qualitative theory might provide a better idea to understand the issue” (p. 210).

The future of mixed methods methodology

As the third known approach to research in academia and industry, mixed methods designs have seen tremendous growth over the years with several dedicated journal publications and textbooks. The launch of the Journal of MMR and various webinars on mixed methods studies to researchers and graduate students worldwide have given more hope to the mixed methods community (Creswell, 2016; Mertens et al., 2016). Experts in the field of mixed methods research have continued to work on the central premise that “the use of qualitative and quantitative approaches, in combination, provides a better understanding of research problems than either approach alone” (Creswell and Plano Clark, 2007: 5). In order for the mixed methods fraternity to maintain the plethora of publications coming from various academic disciplines and industries, experts continue to advance the definition and scope of mixed methods, while solving identified problems related to the quality of integration (see Creswell and Plano Clark, 2011; Kieser et al., 2015; Mertens, 2015a; Teddlie and Tashakkori, 2009). For instance, mixed methods researchers encourage the use of a transformative paradigm that compels investigators to engage in data collection to determine the focus of the investigation and create a collaborative link with community members to ensure sustainable development on the issue of concern (Mertens et al., 2016).

One area of research that will continue to make mixed methods relevant is wicked problems (complex social problems) that involve multiple interacting systems and, therefore, requires a team of experts to address (McChesney, 2020; Mertens, 2015b; Mertens et al., 2016). Complex social issues are problems that require multidisciplinary teams to share their expertise in respectful ways to find answers and solutions, which align well with the core functions of mixed methods studies. A report of the Mixed Methods International Research Associations’ Task Force entitled “the future of mixed methods” has indicated several projects that the Association is working on to strengthen the core areas in MMR (Mertens et al., 2016). Some of the recommendations from the report to improve MMR include: the use of more than one method, methodology, approach, theoretical or paradigmatic framework, and the integration of findings from different components. Again, the use of complex designs to address wicked problems in society has received attention in recent times due to the many components that come together within a given framework to solve such issues (Creswell, 2016; Mertens et al., 2016). For instance, Puschel and Thompson (2011) studied mammogram screening in South America using a mixed method approach. In their study, they initially gathered qualitative data from the women about barriers to screening followed by a quantitative data by a randomized controlled trial that integrated womens’ meaning to access. Geo-coding and the use of geographical information are other forms of innovative approaches for evaluating links between public health data and environment. Geo-coding and the use of geographical information systems are crucial research tools that have found its way into mixed methods designs to help find solutions to complex health and social problems (Baldovin et al., 2015; Creswell, 2016).

With all these essential components of mixed methods sighted in the reports for reinforcement, the future of mixed methods looks promising and will continue to grow in the coming years.

The future of mixed methods also relates to provisions for novice researchers interested in using the methodology. Munce and Archibald (2017) after a review of personal experiences in two Canadian universities, proposed mentorship and training as two key areas to help advance the impact of MMR. They contended that providing exposure to MMR through mentoring and team experience between early career researchers and experienced researchers could help prepare the former for success within the field. Their stance is influenced by the role mentorship plays in related areas such as research productivity, faculty retention and advancement as well as job satisfaction as documented in the literature (Archibald, 2016; Munce and Archibald, 2017). Similarly, they projected that the inclusion of mixed methods as a course in university programs, especially at the undergraduate level could be very helpful (Archibald and Munce, 2015). It is observed that mixed method is seldom taught at the undergraduate level. As stated by Roberts and Allen (2019), several existing textbooks on mixed method research designs are silent on the teaching of the concept at the undergraduate level. Putting it aptly, Mertens et al. (2016) intimated that “quantitative research methods are foundational to the traditional undergraduate social sciences curriculum; not so qualitative or MMR. Consequently, undergraduate and graduate students can graduate having learned little or nothing about qualitative or MMR” (p. 19). In furtherance to this, Archibald and Munce (2015) observed in some Canadian universities that even at the post graduate level, only a few institutions had MMR in their curriculum. Whilst bemoaning the limited existence of mixed method courses, they also cautioned that both qualitative and quantitative methods should be well ingrained into the contents of undergraduate programs before any such introduction of mixed method courses are made (Munce and Archibald, 2017). Roberts and Allen (2019) also added their voice to the need for advanced training of students in mixed methods designs to be part of existing educational curricula observing that it was mostly at the doctoral level that the concept got introduced to upcoming researchers. Others also place the advancement of this third force of research methods on the shoulders of researchers. To Mertens et al. (2016), it starts with the ability of mixed method researchers to publish in high impact journals as well as indulging in greater levels of social activism for positive social change. Overall, getting emerging researchers to have confidence in this methodology coupled with all the good works being done by the MMR fraternity will go a long way to strengthen the position of mixed research as the third paradigm in research

Conclusion

In summary, mixed methods research has come to stay. It takes its foundation from the pragmatic paradigm and seeks to use “what works best” to address complex research problems. Mixed methods work to harness the strengths of the quantitative and qualitative approaches to engender a better understanding of complex phenomena. In so doing, however, several controversies have emerged in the literature. In this paper, we have explored some of these controversies in MMR including the overriding influence of positivist ideologies, the artificial separation of the quantitative and qualitative approaches, the issue of integration, as well as validity issues. We sought to show that most of these tensions and controversies have been the result of poor research and would be addressed if researchers clearly showed epistemological stance and paid attention to solid development of the qualitative and quantitative strands during data collection and analysis, and a sound integration process. Our aim here was to use existing literature to encourage emerging researchers to have confidence in mixed methods research and methodologies amidst these controversies. In addition, we believe the future bodes well for MMR. The suitability of MMR to wicked problems, a growing number of journals and mixed methods publications, the campaign to get mixed methods into university curricula, and the number of projects being undertaken by the Mixed Methods International Research Associations’ Task Force will all go a long way to foreground MMR as a relevant methodology in academia and industry.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Joseph Adu

Author biographies

Joseph Adu is a Public Health specialist with broad teaching and research experience in health care delivery. Extensive experience tutoring in Health Sociology, Health Psychology, and Community Mental Health. He is a recipient of the Western University graduate scholarship. Research interests include mental health care delivery and issues hindering community integration of people diagnosed and treated with mental illness, maternal health care, and pregnancy outcomes. Current doctoral research is focused on Exploring Experiences of Familial Mental Illness Stigma Among Individuals with a Diagnosis of Mental Illness in Canada using qualitative methods.

Mark Fordjour Owusu is a beneficiary of the University of Canterbury Doctoral Scholarship. He is a Researcher at the School of Health Sciences, University of Canterbury. He specializes in applying qualitative methods to understand the policy scenario for preventing and controlling noncommunicable diseases in low and middle-income countries.

Ebenezer Martin-Yeboah is interested in young people’s proficiency in accessing, assessing, and applying health information toward good health and wellbeing, a skill he believes is key is key in attaining the United Nation’s Sustainable Development Goal 3. He is a PhD student at Western University, Canada, and is currently applying multiple methods to investigate the sexual and reproductive health information practices of young men. Aside from his earlier training in M.Phil. (Information Studies), M.Phil. (Population and health), and B.Ed. (Population and Family Life) in Ghana, he also has a decade of practice as academic librarian.

Lisbeth Alexandra Pino Gavidia is a specialist in Public Health with experience on the case-based method of learning. She was an honorable recipient of the International Ecuadorian Scholarship for High Performance Students to study the Master of Public Health at Western University in Canada. She also completed a PhD in the Health Promotion field of the Health and Rehabilitation Sciences at Western University. Her research focuses on how single mothers care for themselves and their children during the COVID-19 pandemic using a mix-methods approach.

Sebastian Gyamfi is a Public Health expert with extensive teaching and research in mental health and psychiatric nursing. He is a beneficiary of the Western University graduate scholarship. He has expertise in Health Promotion and Advanced Practice in Mental Health and Stigma. He also works at Parkwood Institute of Research in London Ontario as a Graduate Research Fellow. He does research in Stigma and Mental Illness, Homelessness, Structural Violence, Therapeutic Relationships and Community Integration. His doctoral project focuses on the relationships among predictors and outcomes of stigma stress appraisal in a population of persons diagnosed with a mental illness using structural equation modeling techniques.

References

Archibald

(2016) Investigator triangulation: A collaborative strategy with potential for mixed methods research. Journal of Mixed Methods Research 10(3): 228–250.

Archibald

Munce

(2015) Challenges and strategies in the recruitment of participants for qualitative research. University of Alberta Health Sciences Journal 11(1): 34–37.

Bahari

(2012) Qualitative versus quantitative research strategies: Contrasting epistemological and ontological assumptions. Jurnal Teknologi 52: 17–28.

Baldovin

Zangrando

Casale

, et al (2015) Geocoding health data with Geographic Information Systems: A pilot study in northeast Italy for developing a standardized data-acquiring format. Journal of Preventive Medicine and Hygiene 56(2): E88–E94.

Barbour

(1999) The case for combining qualitative and quantitative approaches in health services research. Journal of Health Services Research and Policy 4(1): 39–43.

Baškarada

Koronios

(2018) A philosophical discussion of qualitative, quantitative, and mixed methods research in social science. Qualitative Research Journal 18(1): 2–21.

Bazeley

(1999) The bricoleur with a computer: piecing together qualitative and quantitative data. Qualitative Health Research 9(2): 279–287.

Bryman

(2004) Social Research Methods, 2nd edn. London: Oxford University Press.

Bulmer

Bales

Sklar

(eds.) (1991) The Social Survey in Historical Perspective, 1880–1940. Cambridge: Cambridge University Press.

10.

Burns

Grove

(2005) Study Guide for the Practice of Nursing Research: Conduct, Critique, and Utilization. Philadelphia, PA: Saunders.

11.

Cohen

Manion

Morrison

(2000) Research Methods in Education, 5th edn. London: Routledge Falmer.

12.

Coyle

Williams

(2000) An exploration of the epistemological intricacies of using qualitative data to develop a quantitative measure of user views of health care. Journal of Advanced Nursing 31: 1235–1243.

13.

Creswell

(2003) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Thousand Oaks, CA: SAGE Publications.

14.

Creswell

(2007) Qualitative Inquiry & Research Design: Choosing Among Five Approaches, 3rd edn. London: SAGE.

15.

Creswell

(2011) Controversies in mixed methods research. The Sage Handbook of Qualitative Research 4: 269–284.

16.

Creswell

(2013) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. London: SAGE publications.

17.

Creswell

(2014) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. London: SAGE Companion. SAGE Publications.

18.

Creswell

(2016) Reflections on the MMIRA the future of mixed methods task force report. Journal of Mixed Methods Research 10(3): 215–219.

19.

Creswell

Plano Clark

(2007) Designing and Conducting Mixed Methods Research. Thousand Oaks, CA: SAGE.

20.

Creswell

Plano Clark

(2011) Designing and Conducting Mixed Methods Research, 2nd edn. Thousand Oaks, CA: SAGE.

21.

De Lisle

(2011) The benefits and challenges of mixing methods and methodologies: Lessons learnt from implementing qualitatively led mixed methods research designs in Trinidad and Tobago. Caribbean Curriculum 18: 87–120.

22.

Denis

LAS

Palen

Anderson

(2014, May) Mastering Social Media: An Analysis of Jefferson County’s Communications During the 2013 Colorado Floods. In ISCRAM.

23.

Denzin

Lincoln

(eds) (2011) The Sage Handbook of Qualitative Research. London: SAGE.

24.

Fàbregues

Escalante-Barrios

Molina-Azorin

, et al (2021) Taking a critical stance towards mixed methods research: A cross-disciplinary qualitative secondary analysis of researchers’ views. PLoS One 16(7): e0252014.

25.

Fetters

Molina-Azorin

(2017) The Journal of Mixed Methods Research starts a new decade: The mixed methods research integration trilogy and its dimensions. Journal of Mixed Methods Research 11(3): 291–307.

26.

Fletcher

LoBiondo-Wood

Haber

, et al (2005) Nursing research in canada: methods, critical appraisal. The Canadian Nurse 101(4): 10.

27.

Flick

(ed.) (2009) The Sage Qualitative Research Kit: Collection. London: SAGE Publications Limited.

28.

Freshwater

(2007) Reading mixed methods research: Contexts for criticism. Journal of Mixed Methods Research 1(2): 134–146.

29.

Giddings

(2006) Mixed-methods research: Positivism dressed in drag? Journal of Research in Nursing 11(3): 195–203.

30.

Giddings

Grant

(2007) A Trojan horse for positivism? A critique of mixed methods research. Advances in Nursing Science 30(1): 52–60.

31.

Greene

(2012) Engaging critical issues in social inquiry by mixing methods. American Behavioral Scientist 56(6): 755–773.

32.

Guba

Lincoln

(1981) Effective Evaluation: Improving the Usefulness of Evaluation Results Through Responsive and Naturalistic Approaches. San Francisco, CA: Jossey-Bass.

33.

Guba

Lincoln

(1989) Fourth Generation Evaluation. Newbury Park, CA: Sage.

34.

Guba

Lincoln

(1994) Competing paradigms in qualitative research. In: Handbook of Qualitative Research, vol. 2. London: SAGE, pp.163–194.

35.

Guest

(2013) Describing mixed methods research: An alternative to typologies. Journal of Mixed Methods Research 7(2): 141–151.

36.

Hanson

Creswell

Clark

VLP

, et al (2005) Mixed methods research designs in counseling psychology. Journal of Counseling Psychology 52(2): 224–235.

37.

Hathaway

(1995) Assumptions underlying quantitative and qualitative research: Implications for institutional research. Research in Higher Education 36(5): 535–562.

38.

Heale

Twycross

(2015) Validity and reliability in quantitative studies. Evidence-Based Nursing 18(3): 66–67.

39.

Hesse-Biber

Johnson

(2013) Coming at things differently: Future directions of possible engagement with mixed methods research. Journal of Mixed Methods Research 7(2): 103–109.

40.

Hyde

(2000) Recognising deductive processes in qualitative research. Qualitative Market Research An International Journal 3: 82–90.

41.

Ivankova

Kawamura

(2010) Emerging trends in the utilization of integrated designs in the social, behavioral, and health sciences. In: Tashakkori

Teddlie

(eds) The SAGE Handbook of Mixed Methods in Social and Behavioral Research, 2nd edn. Thousand Oaks, CA: SAGE, pp.581–612.

42.

Jebb

Parrigon

Woo

(2017) Exploratory data analysis as a foundation of inductive research. Human Resource Management Review 27(2): 265–276.

43.

Jerabek

(2016) W E B DuBois on the history of empirical social research. Ethnic and Racial Studies 39(8): 1391–1397.

44.

Johnson

Onwuegbuzie

(2004) Mixed methods research: A research paradigm whose time has come. Educ Res 33(7): 14–26.

45.

Johnson

Onwuegbuzie

Turner

(2007) Toward a definition of mixed methods research. Journal of Mixed Methods Research 1(2): 112–133.

46.

Jones

Pereznieto

Presler-Marshall

(2015) How does mixed methods research add value to our understanding of development? In: Hesse-Biber

Burke Johnsonthe

(eds) Oxford Handbook of Multimethod and Mixed Methods Research Inquiry. Oxford: Oxford Library of Psychology, pp. 486–500.

47.

Kieser

Nicolai

Seidl

(2015) The practical relevance of management research: Turning the debate on relevance into a rigorous scientific research program. The Academy of Management Annals 9: 143–233.

48.

Kirk

Miller

(1986) Reliability and Validity in Qualitative Research. Newbury Park, CA: SAGE Publications.

49.

Lincoln

Guba

(1986) Naturalistic Inquiry. Newbury Park, CA: SAGE Publications.

50.

Long

(2017) Validity in mixed methods research in education: The application of Habermas' critical theory. International Journal of Research & Method in Education 40(2): 201–213.

51.

Mathison

(1988) Why triangulate? Educational Researcher 17(2): 13–17.

52.

Maxwell

(2016) Expanding the history and range of mixed methods research. Journal of Mixed Methods Research 10(1): 12–27.

53.

Maxwell

Delaney

(2004) Designing experiments and analyzing data. Mahwah, NJ: Lawrence Erlbaum.

54.

McChesney

(2020) Extending understandings: possibilities and considerations for mixed methods research. New Zealand Annual Review of Education 26: 96–108.

55.

McChesney

Aldridge

(2019) Weaving an interpretivist stance throughout mixed methods research. International Journal of Research & Method in Education 42(3): 225–238.

56.

McKim

(2017) The value of mixed methods research: A mixed methods study. Journal of Mixed Methods Research 11(2): 202–222.

57.

Mertens

(2015a) Mixed methods and wicked problems. Journal of Mixed Methods Research 9(1): 3–6.

58.

Mertens

(2015b) Research in Education and Psychology: Integrating Diversity With Quantitative, Qualitative, and Mixed Methods. Thousand Oaks, CA: SAGE.

59.

Mertens

Bazeley

Bowleg

, et al (2016) Expanding thinking through a kaleidoscopic look into the future: Implications of the mixed methods international research association’s task force report on the future of mixed methods. Journal of Mixed Methods Research 10(3): 221–227.

60.

Miyata

Kai

(2009) Reconsidering evaluation criteria for scientific adequacy in health care research: An integrative framework of quantitative and qualitative criteria. International Journal of Qualitative Methods 8(1): 64–75.

61.

Morgan

(2007) Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research 1(1): 48–76.

62.

Munce

SEP

Archibald

(2017) “The future of mixed methods: A five-year projection to 2020” An early career perspective. Journal of Mixed Methods Research 11(1): 11–14.

63.

Niglas

(2009) How the novice researcher can make sense of mixed methods designs. International Journal of Multiple Research Approaches 3(1): 34–46.

64.

Onwuegbuzie

Johnson

(2006) The validity issue in mixed research. Research in the Schools 13(1): 48–63.

65.

O’Cathain

Murphy

Nicholl

(2007) Why, and how, mixed methods research is undertaken in health services research in England: A mixed methods study. BMC Health Services Research 7: 85.

66.

O’Cathain

Murphy

Nicholl

(2010) Three techniques for integrating data in mixed methods studies. British Medical Journal 314: 1147–1150.

67.

Papadimitriou

Ivankova

Hurtado

(2014) Addressing challenges of conducting quality mixed methods studies in higher education. In: Huisman

Tight

(eds) Theory and Method in Higher Education Research. London: Emerald Group Publishing Limited, pp.133–153.

68.

Perez

(2019) Assessing quality in mixed methods research: a case study operationalizing the legitimation typology. Public Access Theses and Dissertations from the College of Education and Human Sciences. 329. Available at: https://digitalcommons.unl.edu/cehsdiss/329

69.

Puschel

Thompson

(2011) Mammogram screening in Chile: Using mixed methods to implement health policy planning at the primary care level. Breast 20(suppl. 2): S40–S45.

70.

Ritchie

Lewis

Nicholls

, et al (2013) Qualitative Research Practice: A Guide for Social Science Students and Researchers. Thousand Oaks, CA: SAGE.

71.

Roberts

Allen

(2019) A call for the inclusion of mixed methods research in the undergraduate psychology curriculum. Frontiers in Psychology 9: 2709.

72.

Rolfe

(2013) Philosophical Basis for Research. London: Open University Press, pp.11–28.

73.

Schoonenboom

Johnson

(2017) How to construct a mixed methods research design. KZfSS Kölner Zeitschrift für Soziologie und Sozialpsychologie 69(Suppl 2): 107–131.

74.

Schrag

(1992) In defense of positivist research paradigms. Educational Researcher 21(5): 5–8.

75.

Schwandt

(2000) Three epistemological stances for qualitative inquiry: interpretivism, hermeneutics, and social constructionism. In: Denzin

Lincoln

(eds) Handbook of qualitative research. Thousand Oaks, CA: Sage, pp. 189–213.

76.

Shannon-Baker

(2016) Making paradigms meaningful in mixed methods research. Journal of Mixed Methods Research 10(4): 319–334.

77.

Slevitch

(2011) Qualitative and quantitative methodologies compared: Ontological and epistemological perspectives. Journal of Quality Assurance in Hospitality & Tourism 12(1): 73–81.

78.

Tashakkori

Teddlie

(1998) Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA: SAGE.

79.

Tashakkori

Teddlie

(2003b) Issues and dilemmas in teaching research methods courses in social and behavioural sciences: US perspective. International Journal of Social Research Methodology 6: 61–77.

80.

Tashakkori

Teddlie

(2006) Validity issues in mixed methods research: Calling for an integrative framework. In: Paper presented at the annual meeting of the American Educational Research Association, San Francisco, April.

81.

Teddlie

Tashakkori

(2003) Major issues and controversies in the use of mixed methods in the social and behavioral sciences. In: Tashakkori

Teddlie

(eds) Handbook of Mixed Methods in Social & Behavioral Research. Thousand Oaks, CA: SAGE Publications, pp.3–50.

82.

Teddlie

Tashakkori

(2009) Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. Sage.

83.

Tracy

(2010) Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry 16(10): 837–851.

84.

Tuli

(2011) The basis of distinction between qualitative and quantitative research in social science: Reflection on ontological, epistemological and methodological perspectives. Ethiopian Journal of Education and Sciences 6(1): 1–12.

85.

Venkatesh

Brown

Sullivan

(2016) Guidelines for conducting mixed-methods research: An extension and illustration. Journal of the Association for Information Systems 17(7): 435–494.

86.

Whittemore

Chase

Mandle

(2001) Validity in qualitative research. Qualitative Health Research 11(4): 522–537.

87.

Wilkinson

IAG

Staley

(2019) On the pitfalls and promises of using mixed methods in literacy research: Perceptions of reviewers. Research Papers in Education 34(1): 61–83.

88.

Woo

Chae

Jebb

, et al (2016) A closer look at the personality- turnover relationship: Criterion expansion, dark traits, and time. Journal of Management 42(2): 357–385.