Using mixed methods in health research

Abstract

Summary

Mixed methods research is the use of quantitative and qualitative methods in a single study or series of studies. It is an emergent methodology which is increasingly used by health researchers, especially within health services research. There is a growing literature on the theory, design and critical appraisal of mixed methods research. However, there are few papers that summarize this methodological approach for health practitioners who wish to conduct or critically engage with mixed methods studies. The objective of this paper is to provide an accessible introduction to mixed methods for clinicians and researchers unfamiliar with this approach. We present a synthesis of key methodological literature on mixed methods research, with examples from our own work and that of others, to illustrate the practical applications of this approach within health research. We summarize definitions of mixed methods research, the value of this approach, key aspects of study design and analysis, and discuss the potential challenges of combining quantitative and qualitative methods and data. One of the key challenges within mixed methods research is the successful integration of quantitative and qualitative data during analysis and interpretation. However, the integration of different types of data can generate insights into a research question, resulting in enriched understanding of complex health research problems.

Introduction

Mixed methods research is the use of quantitative and qualitative methods in one study. Research is often dichotomized as quantitative or qualitative. Quantitative research, such as clinical trials or observational studies, generates numerical data. On the other hand qualitative approaches tend to generate non-numerical data, using methods such as semi-structured interviews, focus group discussions and participant observation. Historically, quantitative methods have dominated health research. However, qualitative methods have been increasingly accepted by the health research community in the past two decades, with a rise in publication of qualitative studies.¹ As the value of qualitative approaches has been recognized, there has been a growing interest in combining qualitative and quantitative methods. A recent review of health services research within England has shown an increase in the proportion of studies classified as mixed methods from 17% in the mid-1990s to 30% in the early 2000s.² In this paper, we present a synthesis of key literature on mixed methods research, with examples from our own work and that of others to illustrate the practical applications of this approach. This paper is aimed at health researchers and practitioners who are new to the field of mixed methods research and may only have experience of either quantitative or qualitative approaches and methodologies. We wish to provide these readers with an accessible introduction to the increasingly popular methodology of mixed methods research. We hope this will help readers to consider whether their research questions might best be answered by a mixed methods study design, and to engage critically with health research that uses this approach.

Methods

The authors each independently carried out a narrative literature review and met to discuss findings. Literature was identified via searches of PubMed, Google and Google Scholar, and hand-searches of the Journal of Mixed Methods Research, with relevant publications selected after discussion. An important consideration was that papers either had a methodological focus or contained a detailed description of their mixed methods design. For PubMed and Google searches, similar terms were used. For example, the PubMed strategy consisted of title and abstract searches for: ((mixed methods) OR ((mixed OR (qualitative AND quantitative)) AND methods)). We also drew upon recommendations from mixed methods conferences and seminars, and reference lists from key publications.

What is mixed methods research?

The most widely accepted definition of mixed methods research is research that ‘focuses on collecting, analysing, and mixing both quantitative and qualitative data in a single study or a series of studies’.³ Central to the definition is the use of both quantitative and qualitative methods in one study (or a series of connected studies). Separate quantitative and qualitative studies addressing the same research question independently would not be considered ‘mixed methods’ as there would be no integration of approaches at the design, analysis or presentation stage. A recent innovation in mixed methods research is the mixed methods systematic review, which sets out to systematically appraise both quantitative and qualitative literature on a subject area and then synthesize the findings.

Why are mixed methods approaches used?

The underlying assumption of mixed methods research is that it can address some research questions more comprehensively than by using either quantitative or qualitative methods alone.³ Questions that profit most from a mixed methods design tend to be broad and complex, with multiple facets that may each be best explored by quantitative or qualitative methods. See Boxes 1 and 2 for examples from our own work.

Box 1.

Examples of authors’ mixed methods research – JW.

How are general practitioners (GPs) responding to possible child maltreatment in England? A mixed methods study

There is considerable debate about the role that GPs should play in the management of child maltreatment (abuse or neglect). This study aimed to describe and understand the types of responses that GPs were making when faced with a child or family who prompted concerns about child maltreatment. The broad research question about GP responses to child maltreatment prompted several sub-questions; each answered by either a quantitative or qualitative methodology. These sub-questions included:

• How and why do GPs record child maltreatment-related concerns in the electronic health record? (qualitative)⁴

• How frequently do GPs record child maltreatment-related concerns in the electronic health record? (quantitative)?⁵

• Does recording vary over time, by child characteristic and by practice? (quantitative)⁵

• How do primary health care practitioners view the GP’s role in responding to child maltreatment? (qualitative)

• What do primary health care practitioners tell us GPs are doing to respond to children who prompt concerns and why? (qualitative)

We analysed quantitative data from the Health Improvement Network (THIN) UK primary care database and conducted qualitative interviews with GPs, Health Visitors and Practice nurses and undertook observations in primary health care settings. In this study, there were two stages of analysis. First, we analysed the data from each study separately and presented findings from each of the data as answers to the sub-questions. Secondly, we integrated the two data and findings to provide a multi-faceted insight into the broader research question about GP responses to maltreatment. A mixed methods design was chosen to facilitate increased breadth and range of study findings; both illuminated different aspects of the same complex issue. In this case, the two methods allowed access to data and insights that each method alone could not provide. Insights from the mixed methods design included differences between the type of maltreatment concerns that are recorded by GPs in the quantitative dataset and the types of concern that were preoccupying and resource-intensive according to the interviews. The interview and observation data also provided an understanding of a wide range of relevant GP responses, from the perspective of the primary care team, whereas the quantitative dataset could only provide data about recording practices.

Box 2.

Examples of authors’ mixed methods research – ST.

The impact of African ethnicity and migration on pregnancy in women living with HIV in the UK: a mixed methods study

Increasing numbers of HIV-infected women in the UK are becoming pregnant; the majority are Africans. This study aimed to explore outcomes and experiences of pregnancy in migrant African women living with HIV in the UK. This is a complex question encompassing medical and sociocultural factors. Specific objectives included:

• Exploring the association between maternal (i) ethnicity, (ii) African region of birth and (iii) duration of residence in the UK and: timing of antenatal booking,⁶ uptake of antiretroviral therapy in pregnancy, virological suppression at delivery, mother-to-child transmission of HIV, and return for HIV follow-up after pregnancy. (quantitative)

• Exploring possible cultural and socioeconomic factors that may contribute to any identified disparities in clinical outcomes. (qualitative)

• Understanding the experiences of pregnancy and health care systems in migrant African women living with HIV in the UK. (qualitative)

We conducted analyses of national surveillance data followed by semi-structured interviews with pregnant African women living with HIV and their health care providers.⁷ We supplemented interview data with ethnographic research in a charity supporting people living with HIV and an African Pentecostal church in London. Each type of data was analysed separately with findings from one analysis informing the other. Data were also compared and contrasted at the interpretation stage. Where appropriate and feasible, the quantitative and qualitative data has been presented in an integrated way, rather than as separate studies. The quantitative phase enabled us to identify potentially important disparities in outcomes and health care access. The qualitative phase allowed us to understand what may be driving these disparities, whilst also identifying previously neglected aspects of pregnancy in this group of women such as stigma within health care settings. This mixed methods approach has resulted in a richer understanding of different aspects of HIV and pregnancy, placing marginalized women’s voices at the centre of the study.

Usually, quantitative research is associated with a positivist stance and a belief that reality that can be measured and observed objectively. Most commonly, it sets out to test an a priori hypothesis and is therefore conventionally described as ‘deductive’. Strengths of quantitative research include its procedures to minimize confounding and its potential to generate generalizable findings if based on samples that are both large enough and representative. It remains the dominant paradigm in health research. However, this deductive approach is less suited to generating hypotheses about how or why things are happening, or explaining complex social or cultural phenomena.

Qualitative research most often comes from an interpretive framework and is usually informed by the belief that there are multiple realities shaped by personal viewpoints, context and meaning. In-depth qualitative research aims to provide a rich description of views, beliefs and meaning. It also tends to acknowledge the role of researcher and context in shaping and producing the data. Qualitative approaches are described as ‘inductive’ as questions are often open-ended with the analysis allowing hypotheses to emerge from data. High-quality qualitative research can generate robust theory that is applicable to contexts outside of the study area in question, helping to guide practitioners and policy-makers.⁸ However, for research that aims to directly impact on policy and practice, the findings of qualitative research can be limited by the small sample sizes that are necessary for in-depth exploratory work and the consequent lack of generalizabilty.

Mixed methods research therefore has the potential to harness the strengths and counterbalance the weaknesses of both approaches and can be especially powerful when addressing complex, multifaceted issues such as health services interventions⁹ and living with chronic illness.¹⁰

There are many reasons why researchers choose to combine quantitative and qualitative methods in a study.^11,12 We list some common reasons below, using a hypothetical research question about adolescents’ adherence to anticonvulsant medication to illustrate real world applications.

Complementarity: Using data obtained by one method to illustrate results from another. An example of this would be a survey of adolescents with epilepsy demonstrating poor levels of adherence. Semi-structured interviews with a sub-group of those surveyed may allow us to explore barriers to adherence.

Development: Using results from one method to develop or inform the use of the other method. A focus group conducted with a group of adolescents with epilepsy may identify mobile phone technology as a potentially important tool in adherence support. We could then develop a mobile phone ‘app’ that reminds patients to take their medication and conduct an intervention study to assess its impact on adherence levels.

Initiation: Using results from different methods specifically to look for areas of incongruence in order to generate new insights. An illustration of this would be a study exploring the discrepancy between reported adherence in clinic consultations and actual medication adherence. A review of case notes may find adherence levels of over 90% in a clinic population; however, semi-structured interviews with peer researchers may reveal lower levels of adherence and barriers to open discussion with clinicians.

Expansion: Setting out to examine different aspects of a research question, where each aspect warrants different methods. We may wish to conduct a study that explores adherence more broadly. A large-scale survey of adolescents with epilepsy would provide information on adherence levels and associations whilst interviews and focus groups may allow us to engage with individual experiences of chronic illness and medication in adolescence.

Triangulation: Using data obtained by both methods to corroborate findings. For example, we could conduct a clinical study measuring drug levels in individuals and documenting self-reported adherence. Qualitative methods such as video diaries may confirm adherence levels.

To this list we would also add political commitment. That is to say, researchers may recognize, and wish to deploy, the strengths of quantitative research in producing generalizable results but may also be committed to representing the voice of participants in their work.

Whatever the reasons for mixing methods, it is important that authors present these explicitly as it allows us to assess if a mixed methods study design is appropriate for answering the research question.^3,13

How is mixed methods research conducted?

When embarking on a mixed methods research project it is important to consider:

the methods that will be used;

the priority of the methods;

the sequence in which the methods are to be used.

A wide variety of methods exists by which to collect both quantitative and qualitative data. Both the research question and the data required will be the main determinants of the methods used. To a lesser extent, the choice of methods may be influenced by feasibility, the research team’s skills and experience and time constraints.

Priority of methods relates to the emphasis placed on each method in the study. For instance, the study may be predominantly a quantitative study with a small qualitative component, or vice versa. Alternatively, both quantitative and qualitative methods and data may have equal weighting. The emphasis given to each component of the study will be driven mainly by the research question, the skills of the research team and feasibility.

Finally, researchers must decide when each method is to be used in the study. For instance a team may choose to start with a quantitative phase followed by a qualitative phase, or vice versa. Some studies use both quantitative and qualitative methods concurrently. Again the choice of when to use each method is largely dependent on the research question.

The priority and sequence of mixing methods have been elaborated in a typology of mixed methods research models. See Table 1 for typology and specific examples.

Table 1.

Examples of studies using mixed methods.

Mixed method design	Study aim	Methods	Value of mixed methods design
Convergent Quantitative and qualitative methods used concurrently and mixed at interpretation stage	To evaluate the Health Foundation's Safer Patients Initiative (SPI) in hospitals in the UK¹⁴	Quantitative analysis of case note and ward survey data. Qualitative analysis of semi-structured interviews (SSI), focus groups and ward observations.	Both data found little impact of SPI whilst qualitative findings suggested that one explanation may be suboptimal implementation and acceptance from staff. The two types of data corroborate one another (no discernible impact of intervention) and qualitative findings provide one explanation for the unexpected lack of SPI impact on outcomes
Explanatory sequential Qualitative methods used to answer ‘why’ or ‘how’ questions generated from preceding quantitative research	To determine what procedures are used in US hospitals to prevent ventilator-associated pneumonia and why¹⁵	Quantitative analysis of survey data from hospital staff followed by SSI with staff from participating hospitals	The interviews offered one explanation for the quantitative findings that some recommended procedures were used more widely than others (influence of nurses and views about strength of evidence). Both data corroborated the pivotal role of nursing staff and collaborative initiatives
Exploratory sequential Quantitative methods used to answer epidemiological questions generated from preceding quantitative research	To identify and quantify factors contributing to the reduction of alcohol use in hepatitis C positive patients¹⁶	Qualitative analysis of interviews, illness narratives and threaded discussions from websites followed by quantitative analysis of a survey	The qualitative phase allowed identification of new factors that influence drinking in this group, which could be tested on a larger population using a quantitative survey. Together, the data revealed differences in motivations between abusing and non-abusing drinkers with hepatitis C and facilitated recommendations about more effective ways to improve adherence to medical advice in these groups
Embedded A small qualitative component embedded in a larger quantitative study or vice versa	To assess the efficacy of a vaginal microbicidal gel on vaginal HIV transmission¹⁷	A randomized controlled trial in with a social science sub-study, comprising in-depth interviews with trial participants and focus groups	The trial found no evidence of an effect of the gel on HIV transmission. Qualitative data demonstrated high levels of acceptability, revealing the gel’s use for sexual pleasure, suggesting adherence to future gels could be increased by framing them in terms of sexual pleasure
Mixed methods Systematic Review (SR) An SR combining both data types	To assess the impact of social interventions on teenage pregnancy rates and their appropriateness for the UK¹⁸	A meta-analysis of quantitative data from controlled trials and systematic review of qualitative studies on teenage pregnancy in England	The meta-analysis of North American data indicated that these interventions were effective. The qualitative review concluded they were likely to be effective and appropriate in a UK setting. Together, the data suggested that there should be a UK policy initiative to invest in these programmes

How is data analysed in a mixed methods project?

The most important, and perhaps most difficult, aspect of mixed methods research is integrating the qualitative and quantitative data. One approach is to analyse the two data types separately and to then undertake a second stage of analysis where the data and findings from both studies are compared, contrasted and combined.¹⁹ The quantitative and qualitative data are kept analytically distinct and are analysed using techniques usually associated with that type of data; for example, statistical techniques could be used to analyse survey data whilst thematic analysis may be used to analyse interview data. In this approach, the integrity of each data is preserved whilst also capitalizing on the potential for enhanced understanding from combining the two data and sets of findings.

Another approach to mixed methods data analysis is the integrative strategy.²⁰ Rather than keeping the datasets separate, one type of data may be transformed into another type. That is to say that qualitative data may be turned into quantitative data (‘quantitizing’) or quantitative data may be converted into qualitative data (‘qualitizing’).²¹ The former is probably the most common method of this type of integrated analysis. Quantitative transformation is achieved by the numerical coding of qualitative data to create variables that may relate to themes or constructs, allowing statements such as ‘six of 10 participants spoke of the financial barriers to accessing health care’. These data can then be combined with the quantitative dataset and analysed together. Transforming quantitative data into qualitative data is less common. An example of this is the development of narrative psychological ‘types’ from numerical data obtained by questionnaires.²²

Potential challenges in conducting mixed methods research

Despite its considerable strengths as an approach, mixed methods research can present researchers with challenges.^23,24

Firstly, combining methodologies has sometimes been seen as problematic because of the view that quantitative and qualitative belong to separate and incompatible paradigms. In this context, paradigms are the set of practices and beliefs held by an academic community at a given point in time.²⁵ Researchers subscribing to this view argue that it is neither possible nor desirable to combine quantitative and qualitative methods in a study as they represent essentially different and conflicting ways of viewing the world and how we collect information about it.⁸ Other researchers take a more pragmatic view, believing that concerns about the incommensurability of worldviews can be set aside if the combination of quantitative and qualitative methods addresses the research question effectively. This pragmatic view informs much applied mixed methods research in health services or policy.⁸

Secondly, combining two methods in one study can be time consuming and requires experience and skills in both quantitative and qualitative methods. This can mean, in reality, that a mixed methods project requires a team rather than a lone researcher in order to conduct the study rigorously and within the specified time frame. However, it is important that a team comprising members from different disciplines work well together, rather than becoming compartmentalized.²⁶ We believe that a project leader with experience in both quantitative and qualitative methods can act as an important bridge in a mixed methods team.

Thirdly, achieving true integration of the different types of data can be difficult. We have suggested various analytic strategies above but this can be hard to achieve as it requires innovative thinking to move between different types of data and make meaningful links between them. It is therefore important to reflect on the results of a study and ask if your understanding has been enriched by the combination of different types of data. If this is not the case then integration may not have occurred sufficiently.²³

Finally, many researchers cite the difficulty in presenting the results of mixed methods study as a barrier to conducting this type of research.²³ Researchers may decide to present their quantitative and qualitative data separately for different audiences. This strategy may involve a decision to publish additional work focusing on the interpretations and conclusions which come from comparing and contrasting findings from the different data types. See Box 1 for an example of this type of publication strategy. Many journals in the medical sciences have a distinct methodological base and relatively restrictive word limits which may preclude the publication of complex, mixed methods studies. However, as the number of mixed methods studies increases in the health research literature we would expect researchers to feel more confident in the presentation of this type of work.

Conclusion

Many of the areas we explore in health are complex and multifaceted. Mixed methods research (combining quantitative and qualitative methods in one study) is an innovative and increasingly popular way of addressing these complexities. Although mixed methods research presents some challenges, in much the same way as every methodology does, this approach provides the research team with a wider range of tools at their disposal in order to answer a question. We believe that the production and integration of different types of data and the combination of skill sets in a team can generate insights into a research question, resulting in enriched understanding.

Footnotes

DECLARATIONS

References

Harding

Gantley

. Qualitative methods: beyond the cookbook. Fam Pract 1998; 15: 76–9.

O'Cathain

Murphy

Nicholl

. Why, and how, mixed methods research is undertaken in health services research in England: a mixed methods study. BMC Health Serv Res 2007; 7: 85–85.

Creswell

Plano Clark

. Designing and Conducting Mixed Methods Research, London: Sage Publications Ltd, 2007.

Woodman

Allister

Rafi

. A simple approach to improve recording of concerns about child maltreatment in primary care records: developing a quality improvement intervention. Br J Gen Pract 2012; 62: e478–86.

Woodman

Freemantle

Allister

de Lusignan

Gilbert

Petersen

. Variation in recorded child maltreatment concerns in UK primary care records: a cohort study using the health improvement network (THIN) database. PLoS One 2012; 7: e49808–e49808.

Tariq

Elford

Cortina-Borja

Tookey

. On behalf of the National Study of HIV in Pregnancy and Childhood. The association between ethnicity and late presentation to antenatal care among pregnant women living with HIV in the UK and Ireland. AIDS Care 2012; 24: 978–85.

Tariq

Pillen

Tookey

Brown

Elford

. The impact of African ethnicity and migration on pregnancy in women living with HIV in the UK: design and methods. BMC Public Health 2012; 12: 596–596.

Murphy

Dingwall

Greatbatch

Parker

Watson

. Qualitative research methods in health technology assessment: a review of the literature. Health Technol Assess 1998; 2: 1–274.

Raven

Doran

Kostrowski

Gillespie

Elbel

. An intervention to improve care and reduce costs for high-risk patients with frequent hospital admissions: a pilot study. BMC Health Serv Res 2011; 11: 270–270.

10.

Nicca D, Fierz K, Happ MB, Spirig R. Symptom management in HIV/AIDS Symptom Management in HIV/AIDS: a mixed methods approach to describe collaboration and concordance between persons living with HIV and their close support persons. J Mix Methods Res 2012;3:217–35.

11.

Greene

Caracelli

Graham

. Toward a conceptual framework for mixed-method evaluation designs. Educ Eval Policy Anal 1989; 11: 255–74.

12.

Pope

Mays

. Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ 1995; 311: 42–5.

13.

O'Cathain

Murphy

Nicholl

. The quality of mixed methods studies in health services research. J Health Serv Res Policy 2008; 13: 92–8.

14.

Benning

Ghaleb

Suokas

. Large scale organisational intervention to improve patient safety in four UK hospitals: mixed method evaluation. BMJ 2011; 342: d195–d195.

15.

Krein Sarah

Kowalski

Damschroder

Forman

Kaufman

Saint

. Preventing ventilator associated pneumonia in the United States: a multicenter mixed methods study. Infect Control Hosp Epidemiol 2008; 29: 933–40.

16.

Stoller

Webster

Blixen

. Alcohol consumption decisions among nonabusing drinkers diagnosed with hepatitis C. J Mix Methods Res 2009; 3: 65–86.

17.

Montgomery

Gafos

Lees

. Re-framing microbicide acceptability: findings from the MDP301 trial. Cult Health Sex 2010; 12: 649–62.

18.

Harden

Brunton

Fletcher

Oakley

. Teenage pregnancy and social disadvantage: systematic review integrating controlled trials and qualitative studies. BMJ 2009; 339: b4254–b4254.

19.

O'Cathain

Murphy

Nicholl

. Three techniques for integrating data in mixed methods studies. BMJ 2010; 341: 4587–4587.

20.

Caracelli

Greene

. Data analysis strategies for mixed-method evaluation designs. Educ Eval Policy Anal 1993; 15: 195–207.

21.

Tashakkori

Teddlie

. Mixed Methodology: Combining Qualitative and Quantitative Approaches, Thousand Oaks, CA: Sage, 1998.

22.

Sandelowski

. Combining qualitative and quantitative sampling, data collection, and analysis techniques in mixed-method studies. Res Nurs Health 2000; 23: 246–55.

23.

Bryman

. Barriers to integrating quantitative and qualitative research. J Mix Methods Res 2007; 1: 8–22.

24.

Johnson

Onwuegbuzie

. Mixed methods research: a research paradigm whose time has come. Educ Res 2004; 33: 14–26.

25.

Kuhn

. The Structure of Scientific Revolutions, Chicago: University of Chicago Press, [1996] 1962.

26.

O'Cathain

Murphy

Nicholl

. Multidisciplinary, interdisciplinary, or dysfunctional? Team working in mixed-methods research. Qual Health Res 2008; 18: 1574–85.