Sage Journals: Discover world-class research

Abstract

Mixed methods reviews offer an excellent approach to synthesizing qualitative and quantitative evidence to generate more robust implications for practice, research, and policymaking. There are limited guidance and practical examples concerning the methods for adequately synthesizing qualitative and quantitative research findings in mixed reviews. This paper aims to illustrate the application and use of joint displays for qualitative and quantitative synthesis in mixed methods reviews. We used joint displays to synthesize and integrate qualitative and quantitative research findings in a segregated mixed methods review about male nursing students' challenges and experiences. In total, 36 qualitative, six quantitative, and one mixed-methods study was appraised and synthesized in the review. First, the qualitative and quantitative findings were analyzed and synthesized separately. The synthesized findings were integrated through tabular and visual joint displays at two levels of integration. At the first level, a statistics theme display was developed to compare the synthesized qualitative and quantitative findings and the number of studies from which the findings were generated. At the second level, the synthesized qualitative and quantitative findings supported by each other were integrated to identify confirmed, discordant, and expanded inferences using generalizing theme display. The use of two displays allowed in a robust and comprehensive synthesis of studies. Joint displays could serve as an excellent method for rigorous and transparent synthesis of qualitative and quantitative findings and the generation of adequate and relevant inferences in mixed methods reviews.

Keywords

Mixed methods research synthesis mixed methods reviews joint displays literature review synthesis methods

Introduction

Systematic reviews are essential for analyzing, appraising, and synthesizing research findings to generate evidence for the practice, research, and policymaking.^1,2 Researchers and practitioners across various disciplines use systematic reviews for establishing the knowledge base for a given subject, develop practice guidelines, and examine the state of contradictions, consensuses, and debates on a given subject.^1,3 In interdisciplinary research, systematic reviews are useful for mapping the state of knowledge about various constructs, for evaluating and combining various theories, and comparing research and methods across disciplines.⁴ Systematic reviews can involve analysis and synthesis of quantitative and qualitative research and literature alone or in an integrated manner.⁵ The syntheses of qualitative and quantitative literature alone may be constrained in their relevance because of the inherent limitations of qualitative and quantitative designs.⁶ Therefore, mixed reviews offer a comprehensive approach to synthesis through honing onto the strengths of qualitative and quantitative reviews and offering a bigger picture of any studied phenomenon.⁷ However, to conduct the highest quality mixed reviews, the integration of qualitative and quantitative findings is of utmost importance. There is limited guidance available in the literature about different practical approaches to integrate qualitative and quantitative findings. Therefore, this paper offers joint displays as an effective method.

Background

Mixed methods reviews: Types and importance

Mixed methods reviews are advanced review types that aim to synthesize qualitative and quantitative findings in a single review to answer complex questions.⁷ These reviews are considered more usable and relevant to guide practice and policymaking because of their characteristic strength of combining quantitative and qualitative research.⁸ For example, quantitative reviews or meta-analyses offer information about the effectiveness of a complex intervention. However, mixed method reviews of complex interventions facilitate an understanding of the effectiveness, usability, transferability, and acceptability from the perspective of the implementers and users.⁹ Such reviews allow for the inclusion of diverse forms of the theoretical and empirical literature, thereby increasing the relevance of the synthesized findings for researchers, practitioners, and policymakers.^7,8

Mixed methods reviews are classified as segregated, integrated, and contingent mixed reviews.¹⁰ In segregated reviews, qualitative and quantitative studies are treated as separate methodologies and are analyzed and synthesized separately before their drawn conclusions being integrated. Such types of reviews are useful when qualitative and quantitative findings are complementary to each other. Integrated reviews consider qualitative and quantitative findings as readily transformable; hence their findings can be analyzed and synthesized with similar methods. These reviews are relevant when qualitative and quantitative findings could refute, confirm, or expand the drawn conclusions. The contingent reviews are more complex and involve a cyclic analysis and synthesis of qualitative and quantitative research in a stepwise process. A cyclic analysis of literature is continued until the best possible answers to the posed review questions are achieved.⁷,¹⁰ Despite the nature of the mixed reviews, one essential commonality across these types is that the qualitative and quantitative findings should be integrated in a robust and meaningful manner. Therefore, integration is the cornerstone of rigorous and effective mixed reviews that offer clear implications for practice.^7,11

Integration and presentation of integrated results in mixed methods reviews

Several broad approaches have been proposed for integrating qualitative and quantitative findings in a mixed-methods review. Sandelowski et al.¹⁰ suggested using data transformation, qualitizing, quantitizing, and triangulation. Heyvaert et al.⁷ also suggested using mixed methods research integration approaches to combine the analytical conclusions drawn from qualitative and quantitative literature. They also recommended that findings could be synthesized through the development of a theoretical framework or path analysis. The above discussed approaches are mainly used in the integration of qualitative and quantitative data in empirical mixed methods studies. Therefore, may have limited application in integration of qualitative and quantitative results (i.e., secondary data) from published empirical studies, and their adequate and meaningful presentation as integrated findings.

Some researchers offered additional approaches for the integration of secondary data as well as the presentation of integrated results. For example, Thompson-Coon et al.¹² offered an interweave synthesis approach involving extensive teamwork through debate and discussion to understand the conclusions drawn from separate reviews of qualitative and quantitative articles, and then combining them during the final stages. This approach is focused more on the role and contribution of team rather than articulating the method of data analysis and synthesis. Harden et al.¹³ proposed juxtaposing through matrices, using logic models and conceptual frameworks, analyzing underlying theories, testing hypothesis through sub-group analyses, and qualitative comparative analysis. These approaches except the matrix’s method, are mainly applicable to mixed reviews aimed at generating qualitative evidence for the reviews of interventional studies. Also, these approaches may not be applicable to reviews that do not have an underlying theory or conceptual framework for guidance. Recently, Hong et al.¹⁴ offered three approaches, namely, comparison, connection, and assimilation. In comparison approach, separately synthesized qualitative and quantitative findings are compared, and divergence among both findings is examined. In connection, the results of qualitative and quantitative phases of mixed reviews are connected to each other in a sequential manner (qualitative to quantitative and vice versa) of following a thread approach. Finally, assimilation is achieved through qualitizing and quantitizing findings and merging them at the interpretation stage. Although these approaches are invaluable for achieving integration in mixed methods reviews, more pragmatic approaches are required for effective synthesis and the presentation of integrated results in mixed methods reviews. Therefore, we offer a practical approach that could be used to overcome the limitations and enhance the application of all of the aforementioned approaches.

Purpose

To illustrate the application and use of joint displays for qualitative and quantitative synthesis in mixed methods reviews. First, an overview of joint displays and their value in mixed methods research analyses are described. Second, an overview of the segregated mixed methods reviews, and the application of the joint displays is illustrated.

Joint displays and their significance

Joint displays are tabular or graphical tools or matrices for qualitative and quantitative presentation in mixed methods research studies.^15–17 These are structured frameworks used for integration at the planning, analysis, interpretation, and reporting levels.^17–18 Joint displays are useful because they offer comparison and contrast of qualitative and quantitative data, the drawn conclusions from the individual phases, and the integrated results of a mixed-methods study.^19–20 Joint displays offer a visual and transparent presentation of data and allow the readers to understand the intention, process, and technique of data integration, analysis, and interpretation.¹⁵ The joint displays offer a condensed view of complex integration required in any mixed methods study.²¹

Joint displays should be pertinent to the mixed methods research design and the integration approach.^18,21 Nevertheless, effective joint displays include the raw data (e.g., measures of central tendency or dispersion, values of statistical tests, qualitative quotes and themes, and the interpretations drawn from qualitative, quantitative, and mixed phases of a mixed-methods study.^16,22 Efficient and well-developed joint displays result from iterative and continuous interpretations and reconfigurations of raw datasets and the drawn conclusions.¹⁷ Based on the types of raw data presented, mixed methods research design, and the integration technique, joint displays are usually categorized into various types. The common types of displays are statistics-by-themes, participant selection display, instrument development display, qualitative data-experiment display, theoretical lens display, cross-case comparison display, side-by-side display, interview question display, and generalizing themes display.^15–16 In addition, Younas et al.¹⁸ identified another type of joint display called the graphical theme display. Each of these displays are specific to five integration techniques used in mixed methods research studies. These techniques are connecting (analysis of one dataset and connecting with the second via sampling), building (developing data collection tools or interventions from the findings of one phase), merging (comparing two datasets during results and interpretation), and embedding (linking one dataset to the other via above three listed techniques).¹⁹ Along with the raw data, joint displays also present the conclusions drawn from individual phases (i.e., inferences) and conclusions drawn after integrating both datasets (i.e., as meta-inferences). The meta-inferences are also identified as confirmed, discordant, and expanded findings to explicitly demonstrate how integration was achieved.^19–22 The development of efficient displays requires integrative thinking through the explication of data integration techniques,²³ which enables generating new insights about and interpretations of the data.¹⁷

In various disciplines, joint displays have been effectively used for data integration across a range of empirical mixed methods studies. However, there has been limited use of joint displays in mixed methods reviews. For example, Younas et al.^18,24 noted that joint displays allowed for meaningful integration of data in mixed methods nursing studies and allowed researchers to offer a comprehensive account of integrated data. Younas et al.,²⁴ identified that of 175 nursing mixed-methods studies, 10.9% used joint displays. Fàbregues et al.²⁵ identified 159 mixed methods studies in the field of palliative care, of which 14.2% used joint displays. Joint displays type technique (i.e., Pillar Integration Process) has also been used in systematic reviews to integrate qualitative and quantitative findings and generate greater insights about the literature.²⁶ Younas et al.¹⁸ reviewed 17 mixed methods nursing studies which used joint displays as methods of integration. They noted that researchers effectively used joint displays, but the presentation of joint displays required some improvements. Nevertheless, the use of joint displays was beneficial for integrating data at the interpretation and reporting levels in empirical mixed methods studies. The above-mentioned useful applications of joint displays demonstrate that these are useful frameworks for developing unequivocal linkages between qualitative and quantitative datasets. Given their usefulness in empirical mixed methods studies, joint displays can also allow for nuanced and distinct comparison of secondary datasets (i.e., in reviews) and for exemplifying a visual representation of integrated datasets.^17,22

Segregated mixed methods review of male nursing students

Our segregated mixed methods review aimed: a) to develop a comprehensive of male nursing students’ experiences and challenges during classroom and clinical education and b) to determine the strategies used by students to overcome their challenges and stressors.²⁷ Given this two-fold aim, a segregated review was suitable because both qualitative studies were synthesized to develop an understanding of students' experiences and challenges, and quantitative synthesis was warranted to identify tested and testable strategies for managing academic and clinical challenges. The literature search was performed within PubMed, CINAHL, Science Direct, and Open Grey. The eligible 43 studies were selected based on the following inclusion criteria: a) studies published in English from December 1990 to May 2018 in peer reviewed journals, b) studies including male nursing students as sample and explored their experiences and challenges in educational settings, clinical learning environments, and classroom teaching, b) studies using either qualitative, quantitative, or mixed methods designs. We excluded the dissertations, letters, commentaries, book results, and book chapters because these sources may not provide experiences of students from their own perspective.

Of 43 studies, 36 were qualitative, six quantitative, and one mixed-methods. The quantitative studies used cross-sectional and correlational surveys and the mixed methods study used a convergent design. A wide range of qualitative studies such as case studies, phenomenology, ethnography, and qualitative description were included. These studies were critically appraised using three critical appraisal tools namely, Checklist for assessing the quality of quantitative studies,²⁸ VAKS tool for qualitative studies,²⁹ and mixed methods appraisal tool.³⁰ The quality rating for quantitative studies was judged to be strong, moderate to strong, and moderate to weak for two studies each. Among qualitative studies, three were judged to be weak, six moderate, and 27 strong. The only mixed methods study was judged to be moderate in rating.

For data analysis and synthesis, the following information was extracted into literature summary tables: authors, country of origin, purpose, study setting, design, sample, sampling technique, sample characteristics, data collection instruments, data analysis methods, key findings, and strengths and limitations. The sample size ranged from 3-37 for qualitative studies and 29 to 462 for quantitative and mixed methods. The key strengths of the strongly rated studies included large and random samples, valid reliable tools, multiple recruitment strategies, appropriate statistical analysis, data collection from multiple settings, data and researcher triangulation, clear description of the context and rich qualitative findings, clear discussion about bracketing of personal assumptions, audit trail, and member checking.

We combined the key findings using summary tables, narrative summaries, thematic synthesis, and joint displays. During data synthesis emphasis was given to the findings of strongly and moderately rated studies and weak studies were used to support the interpretation of the synthesized results. The mixed-methods study was synthesized under both qualitative and quantitative studies. After critical appraisal and quality rating of the studies, qualitative synthesis was undertaken using thematic synthesis.³¹ In thematic synthesis, we generated first order (direct codes from the reviewed studies), descriptive (themes resulting after collation of generated codes based on their similarities and distinct meanings), and analytical (more abstract themes developed after the collation of the interlinked and related descriptive themes) themes. This synthesis resulted in the development of 20 descriptive themes to capture students’ challenges and experiences. These descriptive themes were collated into analytical themes, but for this paper, the descriptive themes will be illustrated. The quantitative studies were synthesized using narrative summaries and data transformation.⁷ The separate summary of qualitative and quantitative findings is presented in Table 1 so that readers can better understand the descriptive synthesized findings are listed in Table 2. After separate synthesis and interpretation of qualitative and quantitative findings, two joint displays were developed, illustrating two different stages of data integration. Each of these displays and stages of integration are discussed and illustrated as follows.

Table 1.

Qualitative and quantitative findings of segregated mixed review of male nursing students.

Qualitative analytical themes	Quantitative findings
Respect my learning needs too	Students experienced workload
Which work settings are appropriate: confusion prevails	Student ability to perform assigned nursing tasks was affected by patient preferences and restrictions
Call me a nurse	Students assigned tasks did not match their personal interests
Culture of clinical and educational environments and interactions with peers	Patients were not fully aware of the roles and responsibilities of male nursing students
Male nurses understand us better: need more men in nursing	Students noted a lack of male mentors, female language laden nursing curricula, and no acknowledgement of their distinct caring and communications styles compared to female students
What does it feel like to be a male nursing student: prospects, fears, and uncertainties	Students preferred working in emergency, intensive, and mental health settings by personal choice or pressure or gender discrimination
One does not fit all: Switching from one strategy to another on a contextual basis	Students did not feel welcome in clinical settings and experienced rejection from patients, nurses, and other care providers

Table 2.

Statistics theme display.

Qualitative descriptive synthesized findings & codes	Study frequency	Quantitative synthesized findings	Study frequency
Overarching theme: Students Reported Strategies to Tackle Challenges	NA	Role Strain	04
Appraisal-focused strategies (n = 7)	09	Educational Challenges	05
Social support strategies (n = 2)	03	Clinical Challenges	07
Problem focused strategies (n-8)	09
Overarching theme: Students Experiences and Challenges	NA
Female student-centered teaching strategies (n = 3)	05
Feminine culture in nursing education (n = 5)	09
Working in maternal and obstetrics settings (n = 5)	30
Working in mental health settings (n = 1)	01
Expected to work in fast pace settings and settings where heavy lifting is required (n = 3)	05
Identify conflict (n = 3)	06
Feminine discourse (n = 3)	04
Support from educators in clinical settings (n = 7)	10
Relationship with female colleagues (n = 2)	05
Interactions with hospital staff and nurses (n = 8)	16
Interactions and relationships with patients (n = 2)	03
Interactions with educators in classrooms (n = 8)	08
Effect of male educators (n = 3)	05
Need of male nurses and mentors (n = 1)	09
Uncertainty about public and professional identity (n = 3)	07
Future of male nurses (n = 5)	10
Impact on personal self (n = 9)	20

*n= number of first order codes for each qualitative descriptive synthesized finding.

Use of joint displays in qualitative-quantitative synthesis

Statistics theme-display

This display was used at the first stage of data integration, defined as the integration of the number of qualitative and quantitative studies and the synthesized findings from qualitative and quantitative synthesis. In this display, the statistical data comprised of frequencies of studies (refers to the number of studies that described the descriptive theme) addressing each of the descriptive synthesized findings (second order codes). For the descriptive qualitative synthesized findings, the number of first order codes for each theme were denoted as “n” in parenthesis. The number of first order codes for each theme were different from the study frequency because the later were developed after comparison and merging of first order codes across various studies. The frequency of studies and the number of codes enabled us for comparison and determination of the strength of each synthesized finding and assign weightage to each of the findings (Table 2). The findings with study frequency greater than five were assigned a weightage of 2, and those below this score were assigned weight 1. This assignment of weight allowed in discerning the emphasis that should be placed on each finding during the reporting of the review and drawing final interpretations after integrated analysis and synthesis. This weight assignment offers one way to ensure the validity of the findings drawn from individual and synthesized analyses.

The advantages of this display included: a) identification of the state and extent of literature about each generated theme, b) evaluation and appraisal of separately analyzed studies to examine any unanalyzed or underrepresented finding, and c) development of decision rules for further analysis and interpretation at the second stage of integration. From Table 2, it could be noted that five synthesized findings (both qualitative and quantitative) with scores less than five, were considered unanalyzed or underrepresented. Therefore, the studies were subjected to additional in-depth analysis to confirm if the identified frequencies were accurate. This additional analysis revealed no new frequencies, and the synthesized findings were entered into the second stage of integration.

Generalizing theme display

Prior to developing the second joint display, the synthesized qualitative and quantitative findings were compared to determine if quantitative findings and vice versa supported qualitative findings. This comparison identified ten qualitative descriptive themes (out of 20 themes) which were equally supported with the quantitative data. Therefore, in the generalizing theme display, a comprehensive overview of the ten selected descriptive themes (obtained from thematic synthesis) and the corresponding quantitative findings and data (means, standard deviations, and percentages) were reported in the reviewed quantitative studies. The generalizing findings were reported as inferences drawn after the comparison of qualitative and quantitative results. This comparison resulted in eight confirmed, one discordant, and one expanded inference. The confirmed inferences were those in which quantitative and qualitative data were consistent with each other, discordant inference resulted due to inconsistency of qualitative and quantitative data, and the expanded inference was the one in which quantitative or qualitative findings provided additional insight about the results. The discordant finding was identified when students in the qualitative studies offered more negative and mixed views about their interactions with nurse educators and female colleagues. The expanded finding was a result of additional explanation offered by qualitative studies regarding the fears, uncertainties, and clinical and educational prospects of male students. The quantitative and qualitative studies reported that male nursing students were hopeful about increase in the number of male nurses and job prospects, but the qualitative findings also suggested that students had fears and uncertainties about their recognition as professionals and future job and career prospects. The generalizing theme display is presented in Table 3. The first part of this display offered a pictorial overview of the consistent and inconsistent synthesized findings of the qualitative and quantitative studies and the number of integrated findings. The circle's size in the first part of the display represented the number of synthesized findings from each type of analysis. For example, the quantitative circle was the smallest, and the qualitative was the largest. This size estimation in the display allowed us to be cautious about interpreting the results from the mixed synthesis component and explicitly state the strengths and limitations of the reviewed literature. The second part of this display offers a complete account of the generated inferences and qualitative and quantitative findings.

Table 3.

Generalizing theme display.


Descriptive theme	Qualitative findings	Quantitative findings	Inferences
Working in maternal and obstetrics settings	Male nursing students experienced several challenges in maternal, child health, and obstetrics settings. They experienced patients' lack of trust on their caring, their skills, and even straight forward rejection.	The male students (94.78 ± 20.69) experienced more role strain in the obstetrics area than female students (75.62 ± 22.76) (p = 0.01) (Sherrod, 1991).	Confirmed: Students experience rejection from female patients in maternal and obstetrics settings.
Expected to work in fast pace settings where heavy lifting is required.	Students were expected to work in some specialized units such as intensive care and emergency.	Male students indicated that they would prefer working in emergency (35.1%), intensive care (20.5%), administration (19.6%), OR (17.9%), and education (14.5%) (Okrainec, 1994). Male students wanted to work in ICU, OR, and emergency departments. They also indicated their interest for the position of administrators (5.7%) and educators (22.9%) (Ozdemir, 2008).	Confirmed: Students are expected to work in fast pace settings such as emergency and intensive care and are often only used as the labour and to lift heavy loads.
Interactions with educators in classrooms and relationship with female colleagues	Most of the students shared negative views about their interactions with educators in the classroom and teaching environments. Similarly, mixed views about their interactions with female colleagues. Although, male students received support from female educators and students, they also experienced open discrimination.	In total, 81% male students were satisfied with their interactions with instructors and 79.5% with female colleagues (Okrainec, 1994). In total, 53.6% reported unfavorable attitude of clinical educators and 11.7% reported discrimination from instructors (Eswi et al., 2011).	Discordant: Students are discriminated against by female students and nurse educators
Future of male nurses	Several students reported that they are hopeful that the number of male nurses would increase and there would be more job prospects.	In total, 85% men indicated that they expect an increase in the number of male nurses in future.	Expanded: Students are hopeful about the increase in number of male nurses, but are uncertain about future job prospects because of the challenges outlined above.
Impact on personal self	One of the greatest fears of the students was being accused of in appropriate touch and sexual advances.	In total, 66.7% of students were stressed about causing harm to female patients (Eswi et al., 2011) and 90.1% of students were nervous that patients may accuse them of sexual harassment (O' Lynn, 2004)	Confirmed: Students continuously experience stress and isolation in clinical and educational settings and are fearful of being accused of sexual advances and harassment in clinical settings.
Feminine discourse	Students indicated that there was an excessive use of "she" nurse in exams and nursing textbooks. The educators and patents referred to them as "male nurses"	In total, 82% indicated a female language laden curriculum and 89.1% indicated that faculty made anti-male remarks (O' Lynn, 2004)	Confirmed: Feminine discourse is prevalent in educational and clinical settings and male students declined to be referred to as male nurses. They continuously remind others to call them a nurse.
Female student-centered teaching strategies	The educators often neglected learning needs of male students	In total, 80% each indicated that different requirements for maternal/obstetrics clinicals for both men and women and no acknowledgement or content on difference in communication styles of men and women (O' Lynn, 2004).	Confirmed: Mostly nurse educators do not pay attention to the learning needs of male nursing students.
Need of male nurses and mentors	Students indicated the need for more male students, mentors, and educators	In total, 99.1% indicated a lack of mentorship for male students, 74.5% indicated they were not encouraged to seek support from other male students, and 78.4% indicated that there is a shortage of male students and mentors (O' Lynn, 2004).	Confirmed: Students need more male educators, mentors and male students to lend them support and encouragement during their nursing educational journey.
Working in maternal and obstetrics settings	Students were unable to fully practice in maternal and obstetrics settings and experienced rejection and lack of trust from patients.	Students experienced rejections from patients (53.3%), rejected as male caregivers (25%), and lack of patients' cooperation (15.7%) (Eswi et al., 2011). In total, 61.8% indicated lack of guidance on therapeutic touch and 91.9% did not feel welcomed in clinical settings (O' Lynn, 2004).	Confirmed: Students cannot practice in maternal and obstetrics settings because of several cultural reasons, privacy issues, and upright rejection from female patients.
Social support strategies	To deal with patients’ rejections, students sought support from female colleagues, educators, and staff.	Majority of students indicated that they needed assistance from colleagues (3.55 ± 1.21) (Smith, 2006)	Confirmed: The commonly used strategy to deal with these challenges is seeking support from female colleagues and educators.

Adapted with permission from Younas, Sundus et al., (2019), page 273.

Discussion and implications

The integration of qualitative and quantitative findings is essential for conducting the highest quality mixed methods reviews.^7,11 We offered joint displays as the potential methods for adequate and meaningful integration because of their characteristic feature of displaying and representing raw and synthesized qualitative and quantitative findings.^17,22 We expanded the application of joint displays from empirical mixed methods research to mixed systematic reviews and offered this approach for use in combination with the broad integration approaches such as connection, comparison, assimilation, juxtaposition, logical models, and conceptual frameworks.^7,8,13,14 Joint displays could be used in combination with all of the listed approaches because Guetterman et al.^15–16 noted that the joint display types varied based on the employed integration techniques (connecting, building, merging, and embedding). We demonstrated that joint displays are valuable frameworks for linking the qualitative and quantitative integrated findings in mixed methods reviews.

Fetters¹⁷ discussed that the creation of effective joint displays involves planning, development, and linking. Planning refers to the early thinking about the data collection processes and ensuring that both qualitative and quantitative datasets can be integrated. Development entails discovering connections between both datasets and organizing and revisiting the data for meaningful presentation into a matrix, table, or figure. Various kinds of matrices could be integrated into joint displays. Some of the common types are case by variable (presenting case-based qualitative and quantitative data), profile (presenting qualitative themes with respect to demographic or quantitative variables), similarity (presented themes and construct based data with cross-tabulated quantitative data), and pattern matrices (presenting cross-tabulated qualitative and quantitative data of different groups).²⁰ Linking involves finding commonalities between qualitative and quantitative data and offering explicit associations during the presentation. These three steps apply to the development of joint displays in systematic reviews. For example, we included these steps during proposal development. We analyzed the qualitative and quantitative findings in such a manner that integration through joint displays is optimized. It is essential that the development of joint displays should be incorporated in systematic reviews at the earliest stages and should not be considered an afterthought. Therefore, the interweave synthesis approach¹² could enhance the planning, development, and linking of individual qualitative and quantitative and integrated findings. The interweave synthesis could allow the reviewers to use intersubjective questions and immerse in the individual findings of the qualitative and quantitative review phases¹² before developing joint displays to present their synthesized findings.

An important consideration for the integration of qualitative and quantitative findings in mixed methods reviews is that context of the qualitative findings is not undermined during synthesis and integration, and development of overall inferences.⁶ Explicating the context in the reviews contributes to the overall rigor of the generated findings.^6–7 To maximize the context of the synthesized findings, joint displays could be useful. Guetterman et al.^15–16 and Younas et al.¹⁸ offered various kinds of joint displays which could be used in systematic reviews. Some of the displays which could help maximize the context are participant selection display, theoretical lens display, and cross-case comparison display. There are no hard rules to be followed for joint display development. Instead, any of the listed kinds could be tailored and adapted to meet the review's needs. Therefore, researchers intending to use displays in mixed reviews require to incorporate creative ways to make their findings more meaningful and relevant for the readers, practitioners, and policymakers. Innovative approaches, such as images, diagrams, and charts, can be used in tabular and circular displays to enhance the data presentation and reporting.¹⁸

There are some limitations to the use of joint displays. First, our proposal to used joint displays for mixed methods reviews is based on one systematic review. Therefore, further application and evaluation of the joint displays are warranted to demonstrate their usefulness. The use of joint displays in mixed methods research is still emerging, and more advanced types of displays are yet to be offered. Nevertheless, if innovative approaches are used, the application of joint displays mixed methods reviews could be promising for advanced synthesis qualitative and quantitative findings. Second, the combination of joint displays with any of the broad integration approaches should be further evaluated in mixed reviews.

Conclusions

Integrating qualitative and quantitative synthesized findings and generating relevant inferences for review readers can be a daunting task for researchers in mixed-methods systematic reviews. Inadequate integration could affect the rigor and validity of reviews and result in the generation of irrelevant findings for practice and policymaking. Therefore, we illustrated joint displays as useful, flexible, and innovative frameworks for the synthesis of qualitative and quantitative findings in mixed reviews. Joint displays allow for a comprehensive presentation of raw and synthesized findings of qualitative and quantitative phases of mixed reviews; statistical data, codes, sub-themes, and themes for the findings; and the overarching and generalizable inferences after the synthesis. Adapting and using various kinds of joint displays creatively may allow researchers to present review findings in an eloquently. Therefore, joint displays could be an excellent method for rigorous, contextualized, and transparent, synthesis of qualitative and quantitative findings and the generation of adequate inferences in mixed methods reviews. Joint displays are also simple tools to demonstrate the integration of findings in mixed methods systematic reviews. With innovative tailoring of existing types, more advanced types of displays could be developed.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Ahtisham Younas

References

Clarke

What is a systematic review?

Evid Based Nurs 2011; 14: 64.

Wilson

Research methods: systematic reviews. Evid Based Library Inform Pract 2016; 11: 60–62.

Aromataris

Pearson

The systematic review: an overview. Am J Nurs 2014; 114: 53–58.

Burgers

Brugman

Boeynaems

Systematic literature reviews: four applications for interdisciplinary research. J Pragmat 2019; 145: 102–109.

Grant

Booth

A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J 2009; 26: 91–108.

Pearson

White

Bath-Hextall

, et al. A mixed-methods approach to systematic reviews. Int J Evid Based Healthc 2015; 13: 121–131.

Heyvaert

Hannes

Onghena

Using mixed methods research synthesis for literature reviews: the mixed methods research synthesis approach. Thousand Oaks, CA: Sage; 2016.

Harden

Mixed-methods systematic reviews: integrating quantitative and qualitative findings. Focus 2010; 25: 1–7.

Noyes

Booth

Moore

, et al. Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods. BMJ Glob Health 2019; 4: e000893.

10.

Sandelowski

Voils

Barroso

Defining and designing mixed research synthesis studies. Res Sch 2006; 13: 29.

11.

Heyvaert

Maes

Onghena

Mixed methods research synthesis: definition, framework, and potential. Qual Quant 2013; 47: 659–676.

12.

Thompson-Coon

Gwernan‐Jones

Garside

, et al. Developing methods for the overarching synthesis of quantitative and qualitative evidence: the interweave synthesis approach. Res Synth Meth 2019; 11(4): 507–521

13.

Harden

Thomas

Cargo

, et al. Cochrane qualitative and implementation methods group guidance series – paper 5: methods for integrating qualitative and implementation evidence within intervention effectiveness reviews. J Clin Epidemiol 2018; 97: 70–78.

14.

Hong

Rees

Sutcliffe

, et al. Variations of mixed methods reviews approaches: a case study. Res Synth Meth 2020; 1(6): 795–811.

15.

Guetterman

Creswell

Kuckartz

Using joint displays and MAXQDA software to represent the results of mixed methods research. In: M

McCrudden

Schraw

Buckendahl

(eds) Use of visual displays in research and testing: coding, interpreting, and reporting data. Charlotte, NC: Information Age Publishing, 2015, pp. 145–176.

16.

Guetterman

Fetters

Creswell

JW.

Integrating quantitative and qualitative results in health science mixed methods research through joint displays. Ann Fam Med 2015; 13: 554–561.

17.

Fetters

MD.

The mixed methods research workbook: activities for designing, implementing, and publishing projects. Thousand Oaks, CA: Sage, 2019.

18.

Younas

Pedersen

Durante

Characteristics of joint displays illustrating data integration in mixed‐methods nursing studies. J Adv Nurs 2020; 76: 676–686.

19.

Fetters

Curry

Creswell

JW.

Achieving integration in mixed methods designs – principles and practices. Health Serv Res 2013; 48: 2134–2156.

20.

Bazeley

Integrating analyses in mixed methods research. Thousand Oaks, CA: Sage 2018.

21.

Buchholtz

Planning and conducting mixed methods studies in mathematics educational research. In: G

Kaiser

Presmeg

(eds) Compendium for early career researchers in mathematics education. ICME-13 monographs. Cham: Springer, 2019, pp. 131–152.

22.

Creswell

Plano Clark

VL.

Designing and conducting mixed methods research. 3rd ed. Thousand Oaks, CA: Sage, 2018.

23.

Plano-Clark

VL.

Meaningful integration within mixed methods studies: identifying why, what, when, and how. Contemp Educ Psychol 2019; 57: 106–111.

24.

Younas

Pedersen

Tayaben

JL.

Review of mixed-methods research in nursing. Nurs Res 2019; 68: 464–472.

25.

Fàbregues

Hong

Escalante-Barrios

, et al. A methodological review of mixed methods research in palliative and end-of-life care (2014-2019). Int J Environ Res Pub Health 2020; 17: 3853.

26.

Johnson

Grove

Clarke

Pillar integration process: a joint display technique to integrate data in mixed methods research. J Mix Methods Res 2019; 13: 301–320.

27.

Younas

Sundus

Zeb

, et al. A mixed methods review of male nursing students' challenges during nursing education and strategies to tackle these challenges. J Prof Nurs 2019; 35: 260–276.

28.

Kmet

Lee

Cook

LS.

Standard quality assessment criteria for evaluating primary research papers from a variety of fields, https://www.ihe.ca/publications/standard-quality-assessment-criteria-for-evaluatingprimaryresearch-papers-from-a-variety-of-fields (2004, accessed 17 December 2020).

29.

Schou

Hostrup

Lyngso

, et al. Validation of a new assessment tool for qualitative research articles. J Adv Nurs 2012; 68: 2086–2094.

30.

Pluye

Gagnon

Griffiths

, et al. A scoring system for appraising mixed methods research, and concomitantly appraising qualitative, quantitative and mixed methods primary studies in mixed studies reviews. Int J Nurs Stud 2009; 46: 529–546.

31.

Thomas

Harden

Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Meth 2008; 8: 45.

Joint displays for qualitative-quantitative synthesis in mixed methods reviews

Abstract

Keywords

Introduction

Background

Mixed methods reviews: Types and importance

Integration and presentation of integrated results in mixed methods reviews

Purpose

Joint displays and their significance

Segregated mixed methods review of male nursing students

Use of joint displays in qualitative-quantitative synthesis

Statistics theme-display

Generalizing theme display

Discussion and implications

Conclusions

Footnotes

Declaration of Conflicting Interests

Funding

ORCID iD

References