Abstract
In the past decade, there has been a groundswell of interest in the use of mixed-methods approaches to conduct research in the health sciences. However, there remains a paucity of diverse teaching materials, curricula and activities to support the continued expansion of education and innovation in mixed-methods research. Here, we report the development and evaluation of an open-space learning activity and tool to aid teaching the concept of synthesis in mixed-methods research. We detail the iterations of the teaching activity and tool as they were developed, we report student feedback, and we discuss the utility of the activity and tool for introducing the concept of synthesis in mixed-methods research within health science and related fields.
Introduction
The use of mixed-methods research in the health sciences has been gaining ground steadily over the past decade (Creswell and Plano Clark, 2011; Fetters and Freshwater, 2015; O’Cathain et al., 2007). Mixed-methods research approaches to scientific enquiry lend themselves to pragmatic perspectives on answering research questions (Johnson et al., 2007; Johnson and Onwuegbuzie, 2004). Teaching health science students to become health scientists requires in-depth reflection and critical examination of multiple perspectives – how valid perspectives are, who holds them, why they matter and according to whom. Mertens (2010) illustrates the pitfall of the ‘un-examining researcher’: ‘If researchers do not acknowledge (or know) the philosophical assumptions that underlie their works, this does not mean that they have no philosophical assumptions. It merely means that they are operating with unexamined assumptions’ (p. 9).
In this article, we describe an approach to teaching mixed-methods research using a theory-building activity. We describe two iterations of implementing and evaluating the teaching activity, in June 2015 and January 2016. We discuss our current best practice to date, having reflected on the challenges and successes of teaching the concept of synthesis in mixed-methods research.
The current evidence base for pedagogy in mixed methods
Until recently, the availability of literature on teaching mixed-methods research had been lacking, to the extent that ‘those interested in teaching mixed methods research simply have no place to turn for empirically derived evidence concerning what constitutes effective mixed methods research praxis’ (Christ, 2009: 294). There now exists a small but growing body of literature to draw on for the design of mixed-methods research courses, most of which focuses on the rationale and conceptual basis for mixed-methods research studies or the development of curriculum and syllabus design (Creswell et al., 2003; Frels et al., 2014; Guetterman, 2016; Ivankova and Plano Clark, 2018; Onwuegbuzie et al., 2011).
In terms of practical teaching methods, there is little direction in the way of materials and strategies for teaching, but this area is growing (Christ, 2009; Earley, 2007; Hesse-Biber, 2015; Ivankova and Plano Clark, 2018). Recently, there have been a small number of publications documenting educators’ experiences of teaching mixed-methods research and some of the tools and strategies utilised (Hesse-Biber, 2015; Ivankova and Plano Clark, 2018; Poth, 2014); however, the focus has mainly been on the social sciences, rather than on health sciences (Guetterman, 2016) (although it is accepted that there is a large amount of cross-over between these two disciplines). Those writing about such experiences in the mixed-methods research classroom highlight the basis for their teaching strategies as experiential student-led designs, space for reflection, opportunities to take risks, group work, and comparison of conclusions drawn from mixed-methods research data and how conflicts are resolved (Bazeley, 2003; Hesse-Biber, 2015).
There still exists a paucity of literature for both empirical evidence of effective tools or activities for teaching mixed-methods research, and reflective accounts of teachers’ experiences of what works well and what does not in the mixed-methods research classroom (Mertens et al., 2016).
Core concepts in teaching in mixed methods in health research
In a study of pedagogical approaches to teaching mixed-methods research, the research philosophy of the teaching team was highlighted as a determinant of the conceptual basis for a mixed-methods research course (Frels et al., 2014). Hence, the conceptual basis for the course described in this article is dialectic; we support the view and teach from the perspective that multiple paradigms in a mixed-method research study results in an enhanced understanding of the phenomena under scrutiny (Teddlie and Tashakkori, 2009).
Despite the presence of several definitions or understandings of what ‘mixed-methods’ research is, there appears to be agreement on a number of fundamental learning goals for students of mixed-methods research. In a mixed-methods study of mixed methodologists’ approaches to teaching, the following common learning goals were identified as expectations within a mixed-methods research course: critical, practical and creative thinking; integration, for example, integration of thinking, multiple perspectives and/or sources and types of data; interaction with others; and perspectives for understanding (Frels et al., 2014).
Synthesis as a threshold concept
Threshold concepts are described as those which act as ‘“conceptual gateways” or “portals” that lead to a previously inaccessible, and initially perhaps “troublesome”, way of thinking about something’ (Meyer and Land, 2005: 373). Quantitative and qualitative epistemologies have traditionally been seen as two distinct disciplines within health research, and as such have developed their own approaches to thinking and practicing. Mixed methods as a research approach brings these two ‘territories’ together, and therefore requires the student to enter and negotiate new conceptual, and at times ‘troublesome’, space. Here, we use the term troublesome in two main ways: (1) a way of thinking and (2) conceptually challenging or destabilising to the learner. For example, troublesome thinking can be viewed as that which confronts established, comfortable epistemologies (in this case, mixed methodological thinking confronts singular methodological thinking). A troublesome concept may be seen as one that a majority of learners do not initially understand or grasp with ease (in this case, mixed-methods research synthesis is the troublesome concept).
Synthesis is a concept of advanced comprehension. It can be defined as the combination of elements to form a connected whole, which generates new insight about the information being combined. We see it as distinct from summary, which can be defined as a brief statement or account of the main points of something. Synthesis in mixed-methods research is often a troublesome concept for students to grasp or accept, particularly if they come from engrained, exclusively quantitative or qualitative epistemological backgrounds. In mixed-methods research, synthesis is a critical conceptual foundation and anchor point from which students build their subsequent learning. Once negotiated, it transforms students’ understanding of (1) the mixing of methods and (2) the generation and crystallisation of new insights emanating from mixed-methods research findings. It is the authors’ experience that students who struggle to understand and articulate the higher order thinking that occurs in the process of synthesis, often produce work more reflective of summary. An inability to understand and demonstrate the difference between synthesis and summary can act as a barrier for future learning in mixed-methods research. In this way, conceptual synthesis in mixed-methods research features the characteristics of a ‘threshold concept’ (Meyer and Land, 2005). For clarification, in this article we discuss teaching the concept of synthesis, rather than ‘research synthesis’ as a methodological approach to systematic reviewing (Sandelowski et al., 2012).
The key objective of our Synthesis Activity was for students to understand how synthesis was distinct from summary, why synthesis was important in mixed methods and why that was a critical distinction to understand in order to do mixed-methods research well.
Open-space learning
The idea to teach mixed-methods research data synthesis originated from an activity to introduce the concept of theory using ‘Open-space Learning’ (OSL) (Monk et al., 2011). OSL originated from the social sciences and humanities teaching practice; it had much to offer as a means to enhance teaching in transdisciplinary Public Health (Public Health practice and research incorporates epidemiology, sociology, psychology, biology, statistics, medicine and other disciplines). OSL has been described as, In OSL we open public space as well as the private spaces in which we learn. [OSL] explores how it is we can enhance what is fashionably called the student experience of learning. Experience involves risk-taking, it involves experiment, it involves not knowing the outcome of particular areas of exploration, but being willing to take the opportunity that the opening of a space affords them. (Docherty, 2011)
OSL promotes collaborative and active learning using challenging topics and tactile, memorable experiences in open spaces. Space is referred to in both the physical and intellectual senses as spaces which are non-hierarchical, open figuratively and actually, without ‘the reassurance of traditionally arranged furniture’ (Monk et al., 2011: 4). It is a technique with foundations in experiential and kinesthetic learning (Gardner, 1985; Kolb, 1984). Monk describes how OSL can be characterised, with space becoming the following:
Transgressive – traditional barriers are suspended; failure is honoured;
Transitional – work between spaces is in a continual process of forming and re-forming;
Transcendent – OSL moves beyond traditional auditory learning styles;
Transrational – reflective of an intuitive and physical response to understanding as much as a rational one;
Transactional – a free exchange and collectivization of ideas and learning;
Transdisciplinary – stable discipline boundaries are suspended, overruled by participant interaction.
OSL has been tested as a higher education setting in a number of academic fields, including chemistry (Farrer et al., 2010), psychiatry and cultural studies (University of Warwick, 2016). We used OSL as an approach for guiding students through mixed-methods synthesis for the health sciences, basing our teaching on this technique. OSL should not be confused with Open Space Technology, which is a facilitated approach to organising meetings and guiding participants through exploration and reflection on complex concepts and issues (Owen, 2008).
OSL takes a number of forms. Previous examples of OSL-based activities include performance and role-play. One OSL activity to teach the concept of theory uses a series of 10–15 cards with a range of modes of information on them as textual facts, graphical data, images and quotations, developed by Nicholas Monk. Students work in groups to create a narrative with the set of cards they have been given. Topics are usually emotive or controversial, with previous examples including the Death Penalty and Climate Change. We adapted this theory-building activity to reflect topics relevant to the health sciences, and to introduce the concept of synthesis as it is understood in mixed-methods research. We called our activity the ‘Mixed-Methods Synthesis Activity’ and refer to it hereafter as ‘Synthesis Activity’. When we talk about the physical materials we used to teach the Synthesis Activity, we use the term Synthesis Teaching Tool. In this article, we describe our Synthesis Teaching Tool and Synthesis Activity, how we developed them and we describe their evaluation. We reflect on our findings and the establishment of the Synthesis Activity in our mixed-methods research curriculum, and offer insights into teaching challenging mixed-methods research concepts.
Methods
Study design
This article describes the development, pilot and formative evaluation of the Mixed-Methods Synthesis Teaching Tool and Activity over two academic years for the module ‘Mixed Methods for Health Research’ at a UK university. Figure 1 summarises the process of developing, refining and evaluating the Synthesis Teaching Tool and Activity. The development and piloting of the Synthesis Teaching Tool was carried out in the academic year 2014/2015. Following the pilot, the Synthesis Activity was revised, built into the teaching curriculum for the 2015/2016 academic year and evaluated in January 2016. Ethical approval was not sought because the study was classified as evaluation. Student feedback forms were completed at the end of the module and were anonymously submitted. No identifiable personal information is included in the evaluation.
Process diagram of Synthesis Activity development and evaluation.
Description of mixed-method Synthesis Teaching Tool pilot
The teaching team for the pilot session consisted of a research fellow (R.E.J.), a research student (M.M.) and a professor (F.G.). The student body (N = 13) consisted of postgraduate taught students undertaking a Master’s of Science in Health Research, PhD students, and healthcare professionals undertaking continued professional development, making for a varied group in terms of levels of experience and formal research training. All students were required to have experience or training in both qualitative and quantitative methods before admission onto the module. The original aims of the tool were to develop students’ understanding of the concepts of synthesis and integration, hence the learning objectives were formed on what the teaching team felt were the core concepts required for synthesis and integration in mixed-methods research. The team’s prior experiences of teaching allowed for identification of concepts that students often struggled to grasp.
Our aims were as follows:
Engage/enliven/challenge students;
Foster their learning of the concept of synthesis and of integration when mixing different research methods.
To meet these aims, our learning objectives for students were as follows:
Understand the difference between summary and synthesis;
Practice/build confidence in synthesis skills;
Have time and space for reflection and self-examination of assumptions;
Open up new ways of thinking and practicing (via navigation of the threshold concept).
Positioning
The Synthesis Activity was positioned within a teaching session in the middle of the module, following sessions on the introduction mixed-methods research, the basics of mixed method research design, key concepts in mixed method analysis, data transformation and sampling. In the first Synthesis Activity iteration, the activity was delivered within a session on mixed studies reviewing, in order to demonstrate how synthesis is conducted specifically in this context.
Content
In developing the tool, we sought health-based topics for which there was some basis for complexity or debate. The topics selected were intended to enliven students’ interests, encouraging active engagement in the activity. After discussion within the planning group, the topics selected were the 2014 Ebola outbreak in Sierra Leone and a ‘five a day’ fruit and vegetable consumption campaign in the United Kingdom (National Health Service (NHS), 2016). The former topic was considered an emotive subject, which at the time of the activity development, garnered much media attention. The latter topic was the basis of continuous debate within the mainstream UK media. Both were also considered to be scientifically straightforward enough for a multidisciplinary audience coming from a range of backgrounds (e.g. practice-based and academic), but with enough social complexity for debate and discussion.
Materials
Once topics were agreed upon, we sought content which represented a wide range of media (or ‘evidences’) and a wide range of views or interpretations of the topics, from images, screenshots and transcribed video clips, to news articles and ‘below the line’ comments from across a range of news organisations (i.e. tabloids and broadsheets), to graphical presentations of data and abstracts of scientific reports. The purpose of using different types of evidences was to get students to begin rudimentary thinking about mixed-methods research integration and to practice collaborative, interdisciplinary working. An additional consideration was to not be too narrow or thematic in our selection of images – OSL is a student-led, non-directive method, and we wanted to leave figurative and actual space for creative interpretation and for group dynamics to work in developing the ‘output’ and new knowledge.
Space
We used classroom space that was a seminar-style, open plan room (as opposed to a lecture theatre) for the activity. This was a space that the students had been using over the course of the module, which consisted of tables (arranged in small clusters to seat groups of 4–5 students), chairs, a projector and screen and a lectern.
Instruction
Students were asked to work in their small, table-based groups and provided with a set of approximately 15 predesigned A4 (UK letter-sized) cards, which featured images, text, tables and figures of quantitative data, or graphical representations, all themed around the specific topic (Figure 2). They were instructed to lay the cards out on a table or on the floor, and think about how the cards ‘tell a story’ with the data presented. They were encouraged to physically spread out. To clarify the aim of the activity, an unrelated example was presented to the students using a photograph of a nuclear power plant and the following text: This is a picture of a nuclear power plant. The topic may be nuclear power. The theme of the photo could be the politics of power generation, safety, risk assessment, health of environment, health of people, alternative fuel sources, pollution, waste management, over population and energy consumption etc.
Example Activity cards.
The activity leader then described this in more detail. An opportunity for questions and clarification was given. Students were then asked to arrange the activity cards in any way or shape they thought made the most sense for bringing together and telling the story of the data presented on the cards (linear, ‘sun’ shaped, etc.). The activity leader emphasised the following point: that there were a range of different types of ‘data’ (even views from tabloid newspaper comments), because they are perspectives which inform opinions and are reflective of some beliefs, which can matter deeply to the people who believe them. The activity leader gave students physical distance (stepping out of the classroom altogether, disengaging with the students for an initial period of time), and time to work through their activity cards on their own first. This was then followed up by engaging each small group of students through non-participatory observation, clarifying any questions and occasional questioning to support the exploration of the cards. This was particularly useful when groups found the activity, or each other, particularly troublesome, for example, it was challenging for groups to negotiate individual perspectives that were strongly quantitative or strongly qualitative, to establish the value of a range of views and perspectives, and to be confident enough as a group to agree on their view of synthesis (overcoming the fear of getting it ‘wrong’).
Review of the pilot
We reviewed the success of the pilot through informal student feedback and through a formal group reflection by the teaching team conducted immediately after delivering the Synthesis Activity session, led by a predesigned topic guide and recorded on video equipment. The topics covered were as follows:
Student approach How did students interpret and process the activity? How did students work as a group? How did students use the physical space?
Student engagement How well was the activity received by students?
Delivery On a practical level, how well was the activity delivered? How well were the aims of the project met?
Modifications What would we do differently based on these reflections?
Based on this review, the Synthesis Activity session was redesigned for delivery in January 2016. The Synthesis Teaching Tool remained the same.
Formative evaluation
Following the delivery of the January 2016 iteration, a formative evaluation was conducted. This consisted of two elements: student evaluation (quantitative and qualitative) and peer review. Following the evaluation, the teaching team again reflected on the delivery of the Synthesis Activity to inform future iterations and to decide if further changes needed to be made to either the delivery of the Synthesis Activity or the Synthesis Teaching Tool.
Student evaluation
Students provided specific, formal feedback on the activity through a written evaluation at the end of the module. For the quantitative element, students assessed the quality of the sessions and teaching specific to the Synthesis Teaching Tool through rating their agreement with the following statements: (1) ‘this activity was useful in helping to learn about mixed-methods research’; (2) ‘this activity taught me to think about mixed-methods research in a different way’ and (3) ‘The quality of the teaching was… (poor to excellent)’. Students scored their responses on their level of agreement from no agreement with statement (1) to high agreement with statement (10). Qualitative responses were also elicited as free text on the evaluation form.
Peer review
An external academic (teaching fellow) attended the second session as a non-participatory observer and provided in-depth feedback on the delivery of the session and observations of student behaviour. The teaching fellow assessed the quality of and the approach to teaching, drawing on experience and principles of OSL. They were requested to focus on the delivery, engagement and interaction between the teachers and the students during the delivery of the Synthesis Activity session.
Results
Review of pilot
After the June 2015 session, the facilitators met and reflected on the session. Here, we describe the results of the June 2015 session feedback regarding the student approach and engagement, and teacher delivery and modifications. This pilot was then used to modify the January 2016 session.
Student approach
Each student group used a different approach to synthesise the card concepts (Figure 3). Group A (topic: promotion of fruit and vegetable consumption) grouped the cards almost purely at an organisational level, grouping similar ideas together within the cards, but not drawing any conclusions or presenting any overarching themes from the exercise.
Student group activity card configurations Group A, Group B, Group C and Group D.
Group B (topic: promotion of fruit and vegetable consumption) used a similar approach to group A by sorting the cards into groups of similar ideas, but with the addition of considering a framework to organise and understand how the ‘themes’ fit together. The group also discussed the value of different formats of knowledge/evidences, for example, numerical data, photographic images and quotes.
Group C (topic: Ebola outbreak) created a narrative, linear description of the cards, configuring the cards temporally and telling a story from start to finish, of the outbreak.
Group D (topic: Ebola outbreak) positioned the cards in parallel lines, demonstrating two perspectives on the Ebola outbreak, a ‘Western view’ and an ‘African view’. This group provided a unique, in-depth consideration of the issues presented on the cards, and synthesised them to feedback a new interpretation of the knowledge presented. This group presented its findings last and it was through this group that the other groups came to hold a better understanding of the task, and therefore the concept of synthesis versus summary.
Student engagement
Informal feedback from the pilot session in June 2015 revealed that students found the Synthesis Activity in particular useful, and that it helped them understand the difference between summary and synthesis. No other activity-specific student feedback was collected.
Delivery
We reflected that there were some aspects of the session that worked well: the proportion of time given to lecturing and activity; facilitation being non-hierarchical, and open figuratively (self-determined by the student) and actually open (the physical space rearranged) during the activity; and reiterating key learning objectives throughout. There were three aspects of teaching delivery that worked less well in the Synthesis Activity. First, having chairs, bags and coats remaining in the room stifled the ability to move more and engage physically in the space and with each other; for example, we could have removed all of these items completely from the room, making it more physically open in the way OSL espouses. Second, having one member of the group present the findings of their activity at the front of the room resulted in a more traditional feedback process, rather than a fully cooperative one. Third, the activity instruction may have been too ambiguous, creating some of the confusion and reliance on summative technique employed by Groups A and C (i.e. their brief accounts of the main points of the two topics), as opposed to synthesis (i.e. more advanced comprehension, the combination of elements to form a connected whole).
Importantly, we reflected that the strength and purpose of the Synthesis Teaching Tool might be better placed to illustrate mixed-methods research synthesis overall, instead of solely in the context of mixed studies reviewing. We agreed that the Synthesis Activity could therefore introduce more broadly the concept of mixing methods (e.g. via study design) or mixing data (analytical integration). This could then form the introduction of the Mixed Methods in Health Research module and become the basis from which students built their understanding of the concept of mixing qualitative and quantitative approaches. Illustrating the difference between summary and synthesis remained a key concept to convey during the Synthesis Activity.
Modifications
On this basis, we modified our approach in using the Synthesis Teaching Tool for the January 2016 Mixed Methods in Health Research module. Table 1 outlines the modifications made.
Synthesis Activity modifications.
OSL: open-space learning.
The inclusion of a classroom-wide discussion after the activity proved to be valuable to solidifying the importance of differences between summary and synthesis. Student groups presented their thematic/conceptual maps to the rest of the class and gave voice to their thought process and concluding themes. Different themes and explanations for the same Synthesis Teaching Tools clarified contrasting views between student groups, and allowed the difference between summary and synthesis to be reified. The session finished with a palpable increase in understanding of the concept as it applies to mixed-methods research.
Formative evaluation in the January 2016 session
A formative evaluation was undertaken for the delivery of the modified session (Iteration 2), delivered in January 2016. The session was incorporated into the Mixed Methods for Health Research curriculum.
Student feedback
Seven out of nine students returned evaluation forms. Excerpts of student responses to both quantitative and qualitative questions are detailed in Table 2 alongside the learning objectives. In addition to the responses detailed in the table, students rated the quality of the teaching an average of 9.7 on a scale of 1–10 (1 = poor; 10 = excellent).
Student feedback from Iteration 2.
Feedback suggested that students were satisfied with the activity and that it prompted them to think about methodological design differently than they did before. The strong emphasis on visual approaches to design and integration of different types of data was a challenge for some students; however, students reported improvements in their understanding and ability to plan and design mixed-methods research.
Overall, students reported feeling the Synthesis Activity session helped them to (1) visualise and understand the concept of synthesis in mixed-methods research, (2) feel more comfortable in using mixed methods in their research practice and (3) develop skills in the negotiation of different sources of knowledge, beliefs and perspectives.
Peer observation
Delivering the session with the modifications from our first iteration worked well on the day. The external academic observer attending this session commented on how the session ran overall: On the whole, you made this session engaging, dynamic, and relevant for your students. By the end they had clearly enjoyed the session and felt they had taken something from it. The session appeared well planned and ran very smoothly. You also deftly dealt with the fact this was a group that contained a mix of academic backgrounds by reminding them to be respectful in a way that was serious but without being overbearing or patronising and so this was never an issue. You ended on a great note (that ‘multiple modes matter for making sense of a phenomenon’) which perfectly summarised this part of the session and it felt as though the students had thoroughly and practically engaged with this idea.
Discussion
The value of experiential learning
By undertaking this Synthesis Activity, we aimed to expose students to new ‘troublesome’ concepts through a creative, memorable learning experience, in which students were active participants, building new knowledge in a collaborative way. We facilitated the opening up and reframing of their approaches to learning mixed-methods research concepts, and in doing so, created new opportunities for understanding the validity and combination of multiple forms and sources of knowledge.
Experiential learning posits that we learn by doing, that learning comes from critical analysis of experience and acting upon that experience (Kolb, 1984). The Synthesis Activity had an impact on students in that they were challenged to learn in a different way. This challenge resulted in a deeper understanding of a key concept in mixed-methods research for the students.
Negotiating synthesis as a threshold concept
During the Synthesis Activity, the teaching team reflected that some students continued to struggle with the conceptual differences between summary and synthesis. It is characteristic of threshold concepts that students find the newly introduced concept as troublesome (i.e. conceptually challenging or destabilising) (Meyer and Land, 2005; Savin-Baden, 2007), and this was clear in our reflections of students’ approach to the Synthesis Activity during the pilot (Iteration 1).
Concept development for a learner follows a path in which meaning-making transforms from an underdeveloped ‘linear chain-like pattern’ or ‘spoke’ to ‘net-like’ structures as new elements of knowledge and understanding are integrated (Kinchin et al., 2000). We found evidence of this when using the Synthesis Teaching Tool, where the most successful group exhibited a net-like integration of the ideas/narrative they felt was represented by the card content.
By modifying the session following the initial pilot, this crossing of a threshold via a transitional, transgressive and transdisciplinary space became more evident, and was reflected in student feedback from the modified January 2016 session (Iteration 2). Students talked of having ‘a different perspective’, of ‘change’ and ‘broadening own thoughts’ over the course of the week. They noted that analytical integration became clearer, easier and more refined, and that the Synthesis Activity, delivered on the first and the last day, helped them to realise this.
OSL as a technique in teaching mixed-methods research
One of the ideologies of OSL is a wish to ‘match dynamism and originality in research with the same in pedagogy’ (Monk et al., 2011: 117). As a teaching method, this OSL activity offered a chance to replicate the interdisciplinary, collaborative and complex nature of mixed-methods health research and demonstrate its distinctive role in producing/co-creation of new knowledge. For example, one student highlighted that the session allowed them to develop group negotiation skills and become aware of multiple perspectives and interpretations of data.
At the same time, the kinaesthetic nature of the tool and activity resulted in a memorable encounter, and one in which the learning was experienced as much in an ‘embodied sense as… an intellectual one’ (Monk et al., 2011: 118). We anticipated that our modification from the pilot, to make the space more physically open and less hierarchical/traditional, would support this. Student feedback suggests that this was successful, with students describing how useful they found the opening of the physical space and the visual, tactile and creative nature of the Synthesis Activity.
Fundamentally, the Synthesis Teaching Tool and the Synthesis Activity are useful for enhancing the teaching and learning that is congruous/aligned with the philosophy of mixed-methods research, in that it ‘straddles’ the two epistemologies of empiricism and constructivism, enabling both to function simultaneously. For this Synthesis Activity to genuinely reflect OSL principles, this means that empirical evidence is presented to students but they are then asked to co-create meaning from it (Monk et al., 2011). For mixed-methods research, knowledge is often created by integrating ‘empirical’ data with ‘experiential’ data (Johnson and Onwuegbuzie, 2004). OSL is, therefore, a promising approach for exploring the concept of synthesis for students of mixed-methods research.
Application of pedagogical theories
There are elements of several pedagogical theories that can be applied to the development and conduct of this Synthesis Teaching Tool, including cognitivism (which focuses on paced learning to avoid cognitive overload), social constructivism (knowledge is socially and culturally created and is not absolute) and social and situated learning (which emphasises social interaction, with active construction of shared understandings and exploration of alternative viewpoints) (Kinchin et al., 2000; Stewart, 2012). Most closely aligned with the principles of our methods is perhaps the humanist theory. This approach values experiential learning, aims to transform existing understanding and perspectives, and accepts that real-world problems are ‘messy’. The humanist theory suits threshold concepts and OSL well – it concerns transforming understanding/perspectives, student-led learning, challenging and providing an open space/environment (physically and intellectually). It highlights the role of the teacher as one that provides a safe environment for critical reflection (Stewart, 2012). The formative evaluation suggests that this was achieved. The peer observer (teaching fellow) noted that the teaching emphasised a need for students to be ‘respectful’ and student feedback was clearly reflective in nature, with students making statements about fear, challenge, comfort, change and their deep thoughts.
Strengths, limitations and future directions
There are a number of strengths to the way in which we developed and delivered this session. First, we made use of funding available for the advancement of teaching and learning, which provided the teaching team with time to design and develop new materials and conduct a comprehensive evaluation. The funding also allowed employment of a research student, contributing to the professional development of future higher education teachers and providing a student perspective within the teaching team. The activity was based on the tenets of OSL, for which there is a growing evidence base, albeit in different disciplines (University of Warwick, 2016). To our knowledge, this is the first reported attempt to use OSL in a Health Sciences setting, and so forms an additional example upon which future OSL activities can be planned. The evaluation was also a strength in that it was comprehensive, consisting of formalised teaching team reflection, formative student feedback and peer observation (Fry et al., 2015). Evaluating teaching and learning strategies in this structured, reflective way strengthens and enhances the teaching quality (Ashwin et al., 2015; Rowland, 2000). In addition, as our approach aimed for a student-centred learning experience, students’ participation in the planning, implementation and evaluation of their learning was crucial (Ashmore and Robinson, 2014).
An additional benefit of delivering the Synthesis Activity was to discover where each of our students was on the continuum of understanding the concept of synthesis. It allowed us to take stock of their attitudes and abilities during the session, and from this point we could facilitate learning in a more bespoke way. Thus the activity gave us an early insight into the ability of our students and as a result of it we were better able to fine-tune our teaching approach.
Following the modifications made from the pilot and formative evaluation and reflective practice, the Synthesis Activity has been afforded a core position in the mixed-methods research curriculum for health scientists at this higher education institute. It has been taught every year since its development. The Synthesis Activity is used within all the mixed-methods research workshops our team delivers nationally and internationally.
There are a number of limitations in this evaluation. For example, we had difficulty in adapting the space for an OSL activity in a timely manner. The teaching rooms available and much of our additional curricula were traditional classroom environments with ‘clutter’ (tables, chairs, lectern, students’ bags, notebooks, laptops, etc.) Although we made efforts to improve the space following the original pilot, it was impractical to make larger scale changes, such as removing all tables from the room, and as such, the activity does not fully reflect the true nature of OSL. In addition, the session has been delivered to a very specific group of students. Many of the students who undertook the module Mixed Methods in Health Research work in a healthcare setting and are familiar with multidisciplinary styles of working, so the content and delivery of this session may sit more comfortably with them than those from other more homogeneous fields. Consequently, there may have been less conceptual distance for our learners to travel in adapting to an OSL environment and may reflect greater levels of success than others may find. OSL, as a form of experiential learning, acknowledges that all students have prior and concurrent valuable knowledge and experience that contribute to learning, whether they come from professional backgrounds or not (Ashmore and Robinson, 2014; Ashwin et al., 2015). This Synthesis Teaching Tool may, therefore, suit mixed ability and mixed discipline student groups well.
Other teachers of mixed-methods research could adapt the tool or make use of the OSL methods upon which it is based. For example, the Synthesis Teaching Tool could be used in conjunction with a reflective diary for students to monitor their cognitive transitions (the transition they make as a learner, moving across the ‘threshold concept’ boundary and into a deeper understanding of conceptual synthesis for use in mixed-methods research), which could form part of the module assessment. Alternatively, OSL methods could be used as a means of role-play, for example, acting as a participant, or defending a study plan, ethics or grant application to a committee, or similar (Frels et al., 2014).
Contribution towards knowledge of pedagogical approaches in mixed-methods research
By describing, documenting, evaluating and reflecting on our teaching of mixed-methods research in the health sciences, we are contributing towards calls for dialogue between what Tashakkori and Teddlie (2003) describe as ‘first generation’ mixed-methods research teachers, as well as descriptions and reflections on what works well and what does not in teaching mixed-methods research (Earley, 2007). By including student feedback, we have provided additional examples of the student voice lacking in most literature (Onwuegbuzie et al., 2011), with one example from Poth (2014).
Some of the problems we have encountered in teaching mixed-methods research align with those of others teaching in the field, for example, the diversity of the student body (Frels et al., 2012); difficulty in students identifying their philosophical assumptions or bias (Onwuegbuzie et al., 2011); and the confusion felt by students who are beginning to question or switch their disciplinary mental models and paradigmatic stance (Hesse-Biber, 2015). On the basis of our evaluation, we feel that the Synthesis Teaching Tool is suited to a diverse student body, can help students to identify their assumptions and navigate movement through threshold concepts successfully.
The conceptual basis for our tool is also in keeping with current rhetoric around teaching mixed-methods research as an ‘epistemological continuum’ (Niglas, 2007; Onwuegbuzie and Leech, 2005) and appears aligned with others’ expectations of teaching of mixed-methods research in that the goals are for creative, practical and critical thinking, integration and (Frels et al., 2014) examination of students’ philosophical assumptions (Mertens et al., 2016). It provides students with an opportunity to reflect on their personal development as researchers (Earley, 2007). If the aims of teaching are to induct students in discipline-specific ways of thinking and practicing, then this Synthesis Teaching Tool supports students towards mixed-methods research proficiency in the future through fostering some of the characteristics described by Guetterman (2016) as facilitating this endpoint: flexible thinking, a ‘natural’ comfort with mixed-methods research, reflective openness and being part of a discipline that sees the value of mixed methods in research.
We describe here only one component of our broader module ‘Mixed Methods in Health Research’, hence it is difficult to apply an overall pedagogical profile to the specific Synthesis Activity, as suggested by Onwuegbuzie et al. (2011). The Synthesis Teaching Tool could be described as exploratory and conceptually focused (thus, fitting into the Methodological, Conceptual, Exploratory profile), but this does not mean that it would only suit courses described as such. Rather, the Synthesis Teaching Tool could function as an exploratory and conceptually focused element within a course meeting any of the five profiles described in their typology, therein increasing the likelihood that different learning styles are met by a range of pedagogical approaches.
Conclusion
This study demonstrates the successful adaptation of an OSL activity for the teaching of mixed-methods research in health sciences. The revision of the Synthesis Activity from the pilot session, to the January 2016 session, to its final form, demonstrates clear improvements in our approach to the activity based on reflection, student feedback and peer observation, highlighting the benefits of comprehensive and varied evaluation. The Synthesis Teaching Tool enabled the transformation of the learning space conceptually through addressing the troublesome concept of synthesis in a non-traditional way, and physically, via our OSL approach to changing the physical space in which the activity took place. Using this approach facilitated an enhanced teaching and learning experience. We engaged postgraduate students in their learning of mixed-methods research concepts and techniques, providing an opportunity for reflection, confidence building and transformation. We achieved this by using a novel activity to teach a challenging threshold concept (synthesis) and to help students to achieve a core component of becoming an examining researcher, through achieving deeper understanding of the difference between summary and synthesis in mixed-methods research.
Footnotes
Acknowledgements
The authors wish to acknowledge Phillip Gaydon, University of Warwick Teaching Fellow, who provided reflective feedback on our teaching using the Synthesis Teaching Tool. We wish to acknowledge the University of Warwick Institute for Advanced Teaching and Learning (IATL) for funding our Pedagogic Intervention Award. We also wish to thank the student participants who meaningfully engaged with the Synthesis Teaching Tool and Activity and allowed us to assess their engagement.
Author’s note
The views expressed are those of the authors and not necessarily those of the (NHS), the National Institute for Health Research (NIHR) or the Department of Health. Research materials such as the Synthesis Activity may be reasonably requested from the corresponding author. Rebecca E Johnson is now affiliated with School of Nursing, Midwifery, and Health, Coventry University, Coventry, UK and Marie Murphy is affiliated with Institute of Applied Health Research, University of Birmingham, UK.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: All the authors received funding from the University of Warwick, Institute for Advanced Teaching and Learning Pedagogic Intervention Award. R.E.J. and M.M. were supported by an NIHR Collaboration for Research in Applied Health Research and Care West Midlands (CLARHC-WM) grant. No funding source had a role in the evaluation once the grant had been awarded.
Author biographies
Marie completed her PhD on the topic of childhood obesity across ethnic groups in Coventry, in 2018 from the University of Warwick.