Abstract
Background:
Flipped classroom has received much attention during the last few years in China, but inconsistent conclusions are made about the effectiveness this approach in health professions education. This review examined the findings of controlled studies published in Chinese in order to summarize the effects of the flipped classroom methodology. These studies focused specifically on undergraduate level of health professions students.
Methods:
A literature search was conducted using China National Knowledge Infrastructure (CNKI) and Wanfang Data Knowledge Service Platform in June 2019. No date restrictions were used. Peer-reviewed papers were reviewed and experimental studies were included if the study compared student outcomes using flipped classroom method versus traditional lectures. The revised version of the Kirkpatrick’s model was applied to evaluate the effectiveness of the flipped classroom approach.
Results:
In total, 934 articles were obtained. Among them, 235 articles were full text reviewed and 59 met the inclusion criteria. Most of the student agreed that flipped classroom strategy was very helpful improving their self-learning ability, problem solving ability, teamwork, and communication skills, but reported increased workload and less-efficiency compared with traditional lectures. However, the majority of studies (n = 52) showed positive findings of the flipped classroom on changes in knowledge and skills, and 3 in negative. Among these 59 studies, there was little evidence of changes in behavior, organizational practice and patient outcome.
Conclusions:
Better design with reduced content should be considered for instructors to promote learning effectiveness. Long-term effects of flipped classroom should be considered in future studies.
Background
As educators adjust their instructional approach to improve teaching effectiveness and encourage active learning, the “flipped classroom” has become a popular strategy in China. In the flipped classroom (FC), lectures are replaced by pre-class activities, typically viewing online videos, and class time is devoted to interactive activities and discussions. The hypothesis is that FC enables students to learn outside the classroom and promotes active learning and engagement inside the classroom. 1
For a teaching strategy to be considered successful, there must be evidence that students’ learning is enhanced. Supporters of FC methodologies suggest that academic performance is improved when utilizing FC compared with traditional lectures. 2 For instance, Hew and Lo demonstrated that the FC is more effective than traditional classroom in terms of student achievement after analyzing 28 comparative studies. 3 In addition, learning theorists opine that FC strategy follows a learner-centric teaching philosophy, and reframes the faculty-student relationship. 4 More specifically, FC allows students to access recorded lectures and other materials as often as they desire and use class time for interaction and application of content. 5 However, studies showed that the effectiveness of FC may be specific to the content being taught. For example, Ma’s study indicated that lectures should be the major teaching methodology in teaching the theoretical knowledge of biochemistry. 2 Some commentaries have raised concerns about the validity of the assessment methods which evaluate the effectiveness of the FC approach. According to Moffett, educators should choose the appropriate assessment method to evaluate the effectiveness of flipped classroom. 6 As more educational practitioners adopt the FC approach, it is urgent to explore the effectiveness of FC and identify the key factors as well as practical problems that contribute to its effectiveness.
Existing reviews, although limited, suggest that the effects of the flipped classroom on student learning outcomes are inconsistent when compared to traditional lectures.7-9 Chen et al analyzed 9 studies in medical education and found mixed results on student knowledge and skill changes using the flipped classroom compared with the traditional lecture. 7 Gillette et al included 6 studies in pharmacy education and found mixed evidence on student outcomes between lecture and flipped classroom. 8 And Evans compared 24 studies in health professions education and identified 17 that showed a positive effect on student academic outcome when using the FC instead of traditional classroom teaching. 9 It is difficult to generalize about the effects of FC on student achievement due to the limited number of included studies. Further, the search strategy of published systematic reviews that examined the effect of flipped classroom versus lecture have been restricted to studies published in English. While literature on the use of the FC in health professions education has grown rapidly over the last few years in China, the research in Chinese will be able to include more information on the effect of FC compared to lecture in health professions education in the Chinese context. The purpose of this review was to systematically investigate the studies associated with the effectiveness of the FC in health professions education in China. This review was guided by the 2 research questions: (i) What is the scope of the published research on FC in health professions education in China? (ii) What are the effects of the FC in health professions education in China?
Methods
Literature search
This systematic review was carried out based on the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analysis) guidelines. 10 Since this is a literature review, the study does not require ethical approval. We conducted unique searches in 2 Chinese electronic databases: China National Knowledge Infrastructure (CNKI), and Wanfang Data Knowledge Service Platform. The search Boolean was (“flipped classroom”) AND (“health professions education”) to capture all relevant article suggestions. Search was limited to peer-reviewed scholarly sources. Research reported in languages other than Chinese was omitted from this review. No date restrictions were used. This literature search was conducted and completed by the end of June 2019. Articles fell into the time span from the first paper on FC published in 2014 through June 2019. The search strategy was designed with the help of a research librarian from the university.
Review procedure
Figure 1 summarizes the review process and number of articles after each stage. After excluding duplicates, the first 2 authors reviewed the abstracts separately and discussed their findings. If they agreed, the article followed the process; if not, the full article was checked and discussed before being selected. When agreement on the included articles was reached, full articles were reviewed independently by the first 2 authors.
Summary of the review process.
After the articles were identified, the 2 authors (SL and XL) reviewed each article in detail using a rating agreement developed to record information (Table 1). The rating protocol was adapted from Chen et al. 7 Articles which measured the effect of other instruction methods, such as TBL and PBL combined with FC, were excluded. In addition, articles were excluded if no effects of FC were measured. The measure of effectiveness of the FC was categorized based on modified Kirkpatrick’s evaluation model. 11 Based on this model, studies that used controlled studies were further analyzed for effectiveness of the FC to avoid bias. The 2 authors compared their decisions on all inclusions and resolved discrepancies in judgment by discussion. Finally, a total of 59 articles were included in the final analysis.
Variables and categories of rating protocol.
Measures of effectiveness
The studies’ measurement of effectiveness was evaluated according to the revised version of the Kirkpatrick’s model 7 —the 6 hierarchical levels of classification of the outcomes are: (1) learner perception to the intervention; (2a) changes in learner attitude; (2b) changes in learner knowledge and skills; (3) changes in learner behavior; (4a) changes in learner organizational practice; (4b) changes in patient outcome. All 59 articles reported the intervention outcomes. Of the 59 studies, 52 tested the effectiveness of the FC in terms of both learners’ perceptions and knowledge and skills gained. Six studies focused only on the changes on students’ knowledge and skills, while 1 study focused only on the students’ satisfaction with the FC (Table 2).
Summary of controlled studies on FC in health professional education in China (n = 59).
Results
The scope of published research on the FC in health professions education
Since 2014, there have been a growing number of studies on the FC in health professions education in China, with only 3 articles published at that time. In 2015, 13 articles were published, followed by 36 in 2016. The number of publications peaked in 2017 and 2018, with 79 articles published each year. This rising trend may continue as there were already 25 articles published in 2019 by the time this literature search was conducted (Figure 2). Most of the articles were non-empirical research (eg, commentaries or action research).
Publication trend.
Use of FC was evaluated in 17 biomedical domains across the 84 quantitative studies. There were 10 articles in biochemistry, 7 in pathology, 3 in anatomy, and 9 in internal medicine. Nursing and clinical diagnostics had 6 articles each. Radiology, stomatology, histology and embryology, preventive medicine, and surgery had 4 articles each. Anesthesiology, obstetrics and gynecology, and neurosurgery had 2 articles each. Otorhinolaryngology, dermatology, and pediatrics had 1 article each. Fourteen of the research papers were not specific to any particular medical specialty.
Our search included only the undergraduate level of health professions education. Most articles focused on students in the second and third years, and only 5 were focused specifically on students in the clerkship year (Year 4 or 5). In addition, most of the studies focused on students cohorts of less than 100.
The quality of published studies on flipped classroom in health professions education in China
Quality of the published research was evaluated based on the measures of effectiveness. Of the 59 controlled studies, 26 used random selection of participants and 33 used convenience selection.
The FC treatment consisted of pre-class activities and in-class sessions. Pre-class activities consisted of a variety of online methods: videos and other learning materials (PPTs, readings and websites information) to deliver content, and online engagement tools, such as questions, group discussion, case study, assessments and experiments. A few of the studies also provided pre-class orientation or guidelines to help students adapt to the new teaching method. Pre-class online tutorials were available in some interventions. A wide range of in-class activities were represented in the papers and included student group presentations, reviews, case analysis, and discussion.
The effects of flipped classroom
In order to evaluate the effectiveness of the FC on students’ learning, 59 studies using a controlled design were selected for further analysis. Table 2 provides a summary of each study. In the category of controlled studies, 3 distinct designs were recognized: historically controlled study (n = 3); randomized controlled study (n = 26); and non-randomized controlled study (n = 30). The historically controlled studies compared the experimental group to a control group that was offered during a different term or school year. Learners were randomly selected in the randomized controlled studies. The participants were selected by convenience in the non-randomized controlled study (eg, selected by groups). Each of these designs consisted of at least one control group (eg, traditional lecture) and one experimental group (FC). Kirkpatrick’s framework was used to analyze the results of the effectiveness of the FC reported in these 59 studies.
Perceptions of flipped classroom
A total of 53 controlled studies assessed students’ perceptions of the flipped classroom. Among them, 34 of them assessed the satisfaction of the FC group and control group, while 19 of them assessed only FC groups. Most used self-reported satisfaction or self-assessment surveys although 1 used an interview, a teaching quality assessment instrument, and the Dundee Ready Education Environment Measure. Students generally responded favorably to this type of instruction; most of them agreed that FC methodology was very helpful improving their self-learning ability, problem solving ability, teamwork, and communication skills (n = 39). Six studies indicated that students were satisfied with the engagement with faculty members and peers.2,19,20,22,24,31 However, students who received FC instruction reported increased learning load.2,19,23 In addition, feedback from students utilizing FC noted that the instructional method was less efficient compared with traditional lectures.33,51
Knowledge and skills changes with the flipped classroom curriculum
In 58 of the 59 controlled studies, changes in knowledge and skills were assessed as outcome variables. Knowledge test, skills test, and case analysis were most commonly used to document achieved course competencies in these studies (Table 2). To evaluate the effectiveness of FC teaching approach, almost all of the studies applied a statistical t-test with a 95% confidence interval. Research that examined the changes in knowledge and skills of students using FC strategy versus the traditional lecture revealed mixed results; most resulted in positive findings, 3 in negative, and 4 in negligible differences. For example, Wang et al found that students in the FC group performed significantly better on both test and dental sculpting skills (P < .05). 14 Sun et al found that students in the lecture group performed better on physical assessment test than students in the FC group (P < .05). 51 Xiao et al found no significant difference between students in the FC and lecture groups on the test (P < .05). 25 In addition, 25 of the 59 studies were carried out in the clinical learning/courses of medical school and 24 of them evaluated the effects of FC. Among the 24 research, 23 studies examined gain in knowledge and skills when using the FC compared with traditional lecture, and only one of them showed that there was no statistically significant effect between FC and lecture. The effect of the FC on learning has been evaluated across the lower levels of Kirkpatrick’s model: learner perception, changes in learner attitude and changes in learner knowledge and skills. However, among the 59 studies, there was little evidence of changes in behavior, organizational practice and patient outcome.
Discussion
Studies reflected a wide range of implementation strategies for using time before class to present information and time in class for discussion or other forms of student engagement making it difficult to assert broad inferences about education effectiveness. Despite this challenge, we identified 3 key findings in the current FC literature in China. First, since 2014 there has been an increased number of research papers on the effectiveness of the FC approach, with an increase in empiric rather than descriptive studies. In the past 2 years, the quantity of the studies on FC has increased, as well as in the number of the empiric studies. This suggests that not only the quantity of the research increased, but also the quality. This trend needs to be continued in order to develop a strong research basis for the FC method application in health professional education in China. In addition, the empirical research covered almost all the health profession specialties, which indicates that all practitioners in this area were trying to improve student learning experiences. Moreover, contrary to the earlier systematic reviews which focused mainly on research carried out in the pre-clinical learning curricula of medical school, 7 this review may provide more information of the effectiveness of FC in the clinical learning. More studies aimed at clinical years with the patient care environment are needed to explore the evidence of changes in behavior, organizational practice, and patient outcome.
Second, we found that a large proportion of Chinese students prefer the FC teaching method over traditional lectures. Previous studies in health professional education similarly found that more students favored the methodology, especially the self-paced learning before class and increased engagement in class.3,7 However, students also reported FC required a significant time investment before class. The current body of evidence suggests that pre-class work may be excessive. 70 Faculty members may want to consider avoiding student overload - instead of containing excessive detail, pre-class work should cover key points pertaining to in-class content.17,20,24,30,67 Better design with reduced content should be considered for the instructors to promote the learning experiences. For instance, Hew and Lo suggested that the total length of the pre-class video should not exceed 20 minutes. 3 In addition, in-class education design should be connected with pre-class assignments to promote deep learning. Furthermore, student orientation may be required to improve the FC learning experience. By explaining the rationale behind the FC, students may be able to understand the advantages of this form of learning. 71 Guidance and support with the pre-class activities may lead to better student preparation for class and increase the satisfaction rate.
Third, students learning through FC methodology found improved self-learning ability through the pre-class online videos and learning materials, as well as increased communication skills and teamwork through group learning and discussion, enhanced problem solving ability and clinical skills through in-class highly engagement, and enhanced faculty-student and peer communication through the active learning experiences. A systematic review including 29 Chinese articles in nursing education reported that self-directed learning was significantly improved with FC methods compared with traditional teaching. 72 However, few studies included self-regulated learning ability as a variable to evaluate the effectiveness of FC. 73 Future research should consider including self-regulated learning ability as a variable to test the effects of FC, as there is evidence that student preparation is a key factor of effectiveness of the flipped classroom. 8 Moreover, different media should be applied to engage students in a variety of activities. 71 In China, QQ and WeChat are popular platforms and using this kind of media may improve students’ participation and satisfaction.
Fourth, this review indicates a number of positive findings in students’ knowledge and skills from the FC approach. This result is similar to that obtained by Hew and Lo, van Alten et al., and Chen et al.3,7,73 Several studies using a knowledge and clinical or practical skills test, reported increased academic performance with the FC methodology. However, no articles evaluated educational outcomes related to students’ higher order thinking skills, such as critical thinking or creative thinking. There was little study of the long-term improvement in learning outcomes using the FC method compared to lectures. In addition, assessment methods need to align with the desired outcomes to evaluate the effectiveness of a teaching methodology.3,7,33 Critical thinking and problem solving ability are common desired education outcomes of techniques that incorporate active learning yet these traits are not assessed in the current literature. However, existing research primarily relied on knowledge and skills tests to evaluate the learning outcomes. Testing for superficial knowledge acquisition may not sufficient for assessing the FC approach since this teaching method emphasizes application. 3 Future research should consider the relationship between desired outcomes, instructional design, and assessment.
Furthermore, faculty development and technology team support should be implemented to assist faculty members with preparation of pre-class resources and educational design. Researchers found that quality of pre-class videos were an important factor influencing learning, student satisfaction, and completion of assignments,12,15,36,44,68 however many faculty members found video preparation and production to be challenging and time-consuming. Duration of FC curriculum, selection of appropriate subject matter, and student learning skills should be taken into account when designing FC and other curricula.32,52,54 Learning management systems may improve or encourage pre-class video completion rate to enhance the learning effectiveness. 17
This systematic review has several limitations. First, the strict exclusion criteria may have led to the omission of articles. For example, articles that indicated application of the FC but were reinforced by other educational methods were omitted from this review. Second, due to the quantity of research on FC, this review only included undergraduate level of health professional education in China. Third, only 2 independent reviewers were involved in this review. More independent reviewers’ involvement may be considered in future review to improve the validity. Finally, this review only included studies published in Chinese; future reviews with both Chinese and English literature may build a stronger evidence base.
Conclusion
Our review suggests that the FC is a promising teaching method in health professional education in China which may significantly improve learning, especially at the clinical level. In addition, reducing content and enhancing student orientation and support may lead to better a FC learning experience and improved outcomes. Future studies should consider a pre-test to provide a more robust understanding of the nature of change associated with the FC methodology. Longitudinal study could be employed to assess the long-term effects of the FC strategy on student learning. 5 Future investigations should also consider evaluating the higher levels of Kirkpatrick’s outcomes.
Footnotes
Acknowledgements
Not applicable.
Funding:
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Youth Innovative Talents Program of Department of Education of Guangdong Province (Humanities and Social Sciences) No. 2017WQNCX018.
Declaration of conflicting interests:
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Authors Contributions
LS led this study. She searched and screened the literature and drafted and revised the manuscript. LX screened and reviewed the literature. WB and TK contributed to the conception and design of this review and the interpretation of data and revised the manuscript critically for important intellectual content. All authors gave final approval to the version of the paper being submitted.
Availability of data and material
Not applicable.
Ethical approval and consent to participate
Not applicable.
Consent for publication
Not applicable.
