Abstract
Assessment of the fetal heart is a challenging part of any routine obstetrical sonogram. The practice of teaching and learning this skill demands expertise in the visualization and interpretation of normal sonographic appearances of the fetal heart. Recognition of the pathological features associated with commonly seen congenital heart diseases is also very important. In the study reported here, students in an obstetrical sonography course were randomly assigned to two groups. The students in the control group (classroom instruction) on average assessed correctly all five anatomic cardiac features but determined the normal or abnormal status of only 19% of cases, 4.7of 25 ± 4.4 SD. The experimental group (online tutorials) correctly identified 39% of cases, 9.9 of 25 ± 2.7 SD, P < .01. The average score in the experimental group was greater than 84% of the students in the control group, a one sigma effect. These data support the conclusion that students learning to assess the fetal heart for the presence of congenital heart disease using a series of online tutorials and practice exercises compared with students receiving conventional classroom instruction demonstrated an improved ability to correctly identify normal and abnormal fetal heart structures.
Keywords
Introduction
The majority of deaths from congenital defects in childhood can be attributed to congenital heart disease (CHD). 1 The most common cause of birth defects, CHD is most frequently reported with an incidence of 0.8% of all live births. 2 Numerous studies have shown, however, that the prenatal diagnosis of congenital heart disease can improve clinical outcomes of those affected after birth.3–5 Prenatal diagnosis of congenital heart disease is critically important to ensure optimal management of the pregnancy, optimal delivery of the baby, and provision of support for parental decision making. In addition, those at high risk for CHD may be relieved of additional anxiety and fears caused just by the possibility of CHD by the confirmation of normality. 6
The role of the sonographer in prenatal diagnosis of congenital heart disease is to assess the fetal heart in real time, including the four chambers and whenever possible the outflow tracts. 7 Typically the sonographer will document normal or abnormal findings as a preliminary report with corresponding still images. The interpreting physician will review the preliminary report and associated images confirming the determination of a normal, abnormal, or an indeterminate finding.
The formal training for diagnostic medical sonographers in the detection of congenital heart defects is provided in didactic form usually within an obstetrics and gynecology sonography course. The clinical training is most often provided within a practicum through clinical facilities linked to the college program. Imaging and assessing the fetal heart is a challenging task for sonography students. Acquiring images and assessing the fetal heart is a complex task demanding an integrated understanding of the anatomy and physiology of the fetal heart and image recognition skills of rapidly moving structures such as the outflow tracts on the order of 2 to 6 mm in diameter. 8 Unlike abdominal, pelvic, or other sonography exams, students usually do not have any significant opportunity to practice obstetrical sonography in their college imaging laboratories. It is also difficult for education programs to acquire a robust library of sample cases representative of the broad diversity of congenital heart disease.
Addressing complex learning objectives that demand significant skill development from the learner is a challenge for the traditional classroom setting. Skill development demands significant formative assessment and structured feedback, which can be difficult to provide to a group in a classroom environment. For teachers at all levels of education, it remains a challenge to provide customized instruction and high-quality feedback to individuals in large classes.
In a seminal article in 1984, Benjamin Bloom articulated a number of methods to address the two-sigma problem. 9 Bloom found that when using both mastery learning and tutorial-based instruction as compared with conventional classroom instruction, students in the tutorial and mastery learning groups on average increased their achievement levels by two-sigma (two standard deviations). In other words, the average student in the tutorial and mastery group outperformed 98% of the students in the classroom group. Mastery-based instruction proposes that learning should provide formative evaluation with guided or corrective feedback such that students achieve mastery of all of the learning objectives. 10 Tutorial-based instruction assumes that students are working regularly with a good tutor, in a face-to-face mode. Two forms of instruction have emerged to address the opportunity for learning using the theory of the two-sigma problem. 11 These forms include a method of automated feedback based on student performance during practice exercises, and mastery learning, where students work at their own pace to achieve mastery of learning.11,12
In the realm of digitally enhanced education that exists in many postsecondary institutions today, there exists a significant opportunity to provide students with opportunities to leverage the educational theories and applications established by Bloom. Some very popular solutions have emerged from outside the traditional postsecondary sector. Khan Academy is a very popular electronic platform for learning mathematics based on tutorial-based instruction and mastery learning principles. 13 Students are able to traverse the entire spectrum of mathematics necessary to achieve successful preparation for entry into postsecondary institutions. Besides the pedagogical advantages associated with the two-sigma problem, the use of electronic learning tutorials and practice exercises provide numerous advantages compared with traditional classroom learning. These advantages include:
self-paced progress through the learning objectives,
opportunity to repeat tutorials anytime anywhere without reliance on a human tutor,
ability to access experts on niche topics,
expand the learning opportunity to a global audience in any language,
ensure a consistent and high-quality learning opportunity,
provide the tutorial and learning tool designers with valuable quantitative feedback regarding the learning experience that can be used to improve future iterations of the tutorials and tools.
The objective of this study was to determine if a series of online tutorials and practice exercises using a standardized assessment of the fetal heart in real time would be a more effective approach to be used by students when learning how to identify normal fetal heart structures and congenital heart disease. The researchers addressed the following hypothesis
Materials and Methods
This study was approved by the Charles Sturt University Ethics in Human Research Committee, the McMaster University Research Ethics Board (MREB), and the Mohawk College Research Ethics Board prior to initiation. The participants in this study were drawn from the Medical Radiation Science Obstetrical and Gynecologic Ultrasonography 3 class. These sonography students were in the third year of study within a collaborative four-year, Mohawk-McMaster Medical Radiation Sciences program. All 33 students in the class volunteered to participate in this study.
Students were randomly divided into two groups. There were 17 students assigned to the classroom learning (control) group, and 12 of these students completed all of the learning activities. Of the 16 originally assigned to the online learning (experimental) group, 15 students completed all of the learning activities. The 27 students who completed all of the learning activities formed the basis for final data analysis. Three learning objectives for the experimental and control groups were defined as follows:
increased ability to identify normal fetal cardiac structures,
increased ability to identify abnormal fetal cardiac structures,
increased ability to identify abnormal cases.
Students assigned to the control group were not permitted access to the online tutorials and practice exercises. Students assigned to the experimental group were not permitted to attend the four face-to-face classes and single online tutorial with the course instructor. The online examination was administered within two weeks of the onset of the experiment.
Students enrolled in the control group participated in four classroom lecture periods of 50 minutes each and one online tutorial. The lectures reviewed the anatomy of the fetal heart, the fetal blood flow, and the sonographic views used to image the heart. They also included real-time cineloops with tips and suggestions on how to identify a normal heart. Lectures included review of some of the more common congenital heart abnormalities using real-time cineloops. The tutorial was delivered online with labeled anatomy images, with focused questions pertaining to the diagrams. The time for instruction was approximately 240 minutes, with an allowance for independent study time of 90 minutes, for a total of 330 minutes of time committed to learning.
The experimental group was provided with the “Find It First” tool consisting of 13 brief online tutorials in the use of a five-step method of assessment of the fetal heart prepared by the study investigator, amounting to approximately 75 minutes of instruction in total. Students completed practice exercises online as formative assessment with electronic feedback that required an additional 150 minutes of learning time. The total time committed to learning for the experimental group was approximately 225 minutes.
The students were also provided practice cases for each step of the technique, and these exercises included immediate electronic feedback. Students were required to demonstrate mastery of each step before continuing to the next step of assessment. A screen shot depicting the user interface for the practice exercises provided by module two is provided in Figure 1. Students worked through five online modules representing five steps of analysis considering the anatomic features and relationships as follows:
cardiac situs, axis, and size;
chamber morphology and function;
left ventricular outflow tract (LVOT) continuity with the left ventricle;
right ventricular outflow tract (RVOT) continuity with the right ventricle;
cross-over of outflow tracts at their origins.
A screen shot depicting the user interface for the practice exercises provided by module two of the “Find It First” tool. During formative assessment the feedback is provided immediately. All cases that are incorrect must be repeated after the first practice series is completed before continuing to the next learning module.
Each of these features and relationships could be defined as normal, abnormal, or not seen. Each module consisted of a brief tutorial to provide examples and instruction on how to assess the relevant anatomic feature. Following the review of each tutorial the students were required to complete 17 sample cases similar to the example depicted in Figure 1. At the end of every series, all of the missed cases were compiled, and the learner reassessed the missed cases until all 17 cases were accurately reviewed. The student could advance to the next module only after successfully demonstrating the ability to analyze the anatomic features covered by the module. As students advanced through the modules, additional steps were added to the analysis until the student reached the fifth module where all five steps and corresponding features were examined, representing a review and analysis of a total of 85 cases.
In order to assess the performance of the students in both groups an online quiz was developed. Each student was provided with a unique and secure user identification and password. Students’ ability to identify normal and abnormal structures was measured by their review of 25 cases of fetal cardiac anatomy. A randomized sequence of 10 abnormal and 15 normal cases were presented. Each case consisted of two cine-loop sweeps through the fetal heart. The situs to four chamber sweep began in the transverse abdominal plane at the level of the stomach and ended at the four chamber view. The four chamber to outflows sweep began in the four chamber view and swept through the ventricular outflow tracts. Students in both groups completed the online examination within two weeks of completing the classroom or online instruction. As described in Figure 2, students were required to identify five anatomic features for each case and determine whether that case was normal, abnormal, or indeterminate. A completely accurate assessment was one that scored each feature accurately and correctly identified the case as normal, abnormal, or indeterminate. The decision framework was standardized for the students using a protocol that defined the following outcomes: a normal result implies no further assessment, an abnormal determination will result in a referral to a pediatric cardiologist, and an indeterminate finding will result in a follow-up reassessment in the obstetric screening center. This evaluation required approximately one hour for the students to complete.
Screen shot of an example taken from the final quiz. The final quiz includes the assessment of all five indicators and the final decision.
Data Analysis
The chi square test was performed to evaluate the statistical significance of the results of the image analysis. The null hypothesis for this test was defined by the following statement: “After using the computerized learning system (experimental group) or gaining instruction in a traditional face to face classroom environment (control group), there will be no statistical difference in students’ ability to identify normal and abnormal fetal heart structures.” Differences on the multiple choice evaluations and the confidence levels were tested using the two-tailed t-test for statistical significance.
Results
The students were evaluated on their ability to correctly assign one of three conditions, normal, abnormal, or not seen, to five unique anatomic criteria and relationships and correctly define the case as normal, abnormal, or indeterminate. For each case, to be scored as completely correct, all five anatomic criteria and the assignment of normal, abnormal, or indeterminate had to be correct, which required a total of six correct decisions for each case. The students in the control group on average assessed correctly all five anatomic features but determined the normalcy or abnormal status of only 19% of cases, 4.7 of 25 ± 4.4 SD compared with the experimental group mean score of 39% of correctly identified cases, 9.9 of 25 ± 2.7 SD (P < .01). In terms of mean performance difference between the two groups there was a one-sigma (one standard deviation) effect. This can be interpreted to mean that the average score in the experimental group was greater than 84% of the students in the control group. As seen in Figure 3, the experimental group outperformed the control group in every category of assessment.
Five points of assessment of the fetal heart and the percentage of complete examinations correctly identified as normal, abnormal, or not seen. Students in the experimental group (Series 1) used the online tool compared with students in the control group (Series 2) who participated in classroom learning.
The abnormal cases depicted in Table 1 represent a typical range of serious congenital heart disease with one or more abnormal anatomical features evident in the cine-loops provided to the students for review. As shown in the table, the experimental group outperformed the control group in 7 of 10 abnormal cases (P < .01). Specifically in five abnormal cases, D-TGA, single ventricles, large heart, HLHS, and truncus arteriosus, a greater than two-thirds majority of experimental students identified these cases whereas the control group only identified 1 case, truncus arteriosus by a two-thirds majority of students (P < .01). The experimental group performed particularly poorly in detecting Ebstein’s anomaly, displacement of the septal leaflet of the tricuspid valve typically causing regurgitation, with only a 7% detection rate compared with 33% for the control group.
Detection rate by group according to the individual cases of congenital heart disease.
Discussion
The strategy adopted by most sonography programs for teaching students to identify the presence of pathology during abdominal or pelvic examinations is typically based on providing students with a significant amount of supervised practice and exposure to the normal sonographic appearances of anatomic structures. In teaching students to identify the presence of pathology, sonography programs begin with a focus on the normal anatomical appearance of the structure. In this way, students develop expectations in terms of the visualization of relationships and structures that establish normalcy. Once understanding of normalcy is firmly grasped and rooted in the student’s experience, most programs and courses attempt to provide a framework for pathology such that students can focus their attention on links between the patient’s history and the probable associated pathology. For example, a history of right upper quadrant pain and elevated liver enzymes would draw the sonographer’s attention to the liver and biliary tree.
To put this educational paradigm in simpler terms, if the signal is the presence of pathology and the factors contributing to missing the detection of pathology are defined as noise, a sonographer can think about enhancing the signal to noise ratio in order to identify pathology more frequently. To simplify this analogy, if the pathology is the signal, and the factors that influence missed detection of a pathology are the noise, then the sonographer’s task is to enhance the signal to noise ratio in an effort to increase the detection rate of pathology. A deeper understanding of frequent sonographic appearances of pathology relative to patient history will strengthen the signal. The sonographer’s ability to optimize image quality, perform a complete real-time interrogation, and apply systematic examination techniques will help to reduce the noise. In this way, the presence of pathology (signal) can be enhanced during the sonographic examination.
Part of the challenge for detecting congenital heart disease in utero is the fact that most often it is found in the low-risk population. 14 This makes it more difficult to detect because one cannot usually rely on clues from the patient history (signal) to focus the examination. Furthermore, as a screening examination, the inherent bias is an expectation of normalcy. In this setting the ability to reduce noise to enhance what is a weak signal becomes a very important consideration. Standardizing imaging procedures and assessment protocols represents an important consideration in this regard and one that was adopted for this learning tool.
In addition to the two-sigma pedagogy, the online learning tutorials and practice exercises focused nearly exclusively on exposing students in the experimental group to as many normal and abnormal cases as possible in a step-wise learning methodology. The focus for the learner was on practicing the decision making for each step in each module until mastery of the concept was achieved. In the classroom learning group, a broader focus was taken and students were given more contextual information related to the normal and abnormal heart. There was less emphasis on skill development with feedback in the classroom setting.
Bloom’s two-sigma theory was not replicated in this study; however, a one-sigma improvement was found and this represents a twofold increase in the percentage of cases accurately classified by students in the experimental group, 39% compared with 19% in the control group. Further, the experimental group did not benefit from any face-to-face learning opportunities. This might also explain why a two-sigma effect was not achieved. The ability to correctly classify anatomic features and relationships was consistently demonstrated across all five categories. This suggests that students acquired improved image interpretation skills in the experimental group as compared to the control group.
The majority of congenital heart defects (7/10) were identified with greater frequency by the experimental group compared with the control group. In addition, the experimental group was much more consistent as a group in identifying 5 of 10 defects with a greater than two-thirds majority of the group compared with the control group, which only achieved a two-thirds majority correct identification for one abnormal case. These results suggest that the use of mastery-based instruction in the form of practice exercises framed in a step-wise protocol might lead to a more consistent development of image interpretation skills. The particularly poor performance of the experimental group with respect to the Ebstein’s anomaly may be due to the fact that it was not included as an example during the online tutorials and it represents the singular isolated abnormality of an apical displacement of the tricuspid valve. The online tutorials emphasized more frequently encountered structural abnormalities, including presence of ventricular septal defects, discontinuity of the outflow tracts, disparity in chamber size, heart size and angulation, among other factors. Future versions of this learning tool can be expanded to correct this oversight.
This study was limited in a number of ways. The number of participants was relatively small and representative of a single cohort of sonography students. Five of the 17 students in the control group did not complete the study while only one of the students in the experimental group did not complete the study. The students in the experimental group may have persisted through the final evaluation and completed the study in a greater percentage due to their familiarity with the “Find It First” learning tool. Some of the students from the classroom (control) group may have found the final evaluation provided on the “Find It First” tool to be unfamiliar and this may have been enough to deter them from completing the study. The online learning tutorials and practice exercises were first generation and improvements in content and technical terms can be achieved in future generations of this application. Finally, the performance of the students in the control (classroom) group was dependent on a single instructor and this could bias the outcome of the student’s learning in either a positive or negative direction.
Conclusion
In this study it was shown that students’ ability to correctly classify normal and abnormal fetal cardiac anatomy can be improved using principles of mastery learning and tutorial-based instruction in an online format. Further, the opportunity to exploit the benefits of online learning may provide students with greater access to high-quality learning opportunities. Students may benefit from a self-paced approach to learning with the opportunity to review tutorial sessions anytime, anywhere as much as desired.
Assessment of the fetal heart for congenital heart disease is a challenging task and one that demands significant educational expertise in the domains of cardiovascular and obstetrical practice areas. For topics that demand complex analysis and clinical expertise at the intersection of clinical skills such as fetal cardiac assessment, an online tutorial-based approach may prove to be a very effective means of addressing this teaching and learning challenge.
In this study, students using the online tutorial and practice exercises had improved performance by one standard deviation compared with students using traditional classroom-based learning methods. Further examination of this approach, including quantitative analysis of the user experience, may identify opportunities to refine and improve this approach to learning how to assess the fetal heart. Future studies may demonstrate benefits to other stakeholders including practicing clinical sonographers and interpreting physicians. It may prove worthwhile for other educational applications to be developed using this methodology as well, particularly those areas of sonographic practice that are not fully addressed through traditional curriculum. Examples where online tutorial-based instruction might be developed include musculoskeletal imaging, volume imaging, and imaging of the bowel.
Footnotes
Acknowledgements
The authors would like to acknowledge the efforts of University of Hawaii at Hilo Computer Sciences 460 students, Joel Cook, Jacob Shon, Kylie Gonsalves, and Jermaine Vitales for their development of the “Find It First” learning application.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
