Abstract
Introduction
Providers can evaluate patients who sustain trauma during outdoor activities by using the extended focused assessment with sonography for trauma (FAST) and the limited knee ultrasound. Remote tele-mentored ultrasound (RTMUS) can help minimally trained providers in the wilderness if they have difficulty obtaining a view or have questions about the interpretation of an image. The goal of our study was to determine the feasibility of using RTMUS to teach the FAST exam and knee ultrasound exam to ultrasound-naive medical students during a wilderness medicine outdoor activity.
Methods
Medical students from two large academic institutions were randomized to receive either in-person or RTMUS education during a wilderness activity 1 d before completion of FAST and knee ultrasound exams. All students received limited ultrasound instruction before the event. The images obtained by the students were compared. Two ultrasound-trained emergency physicians evaluated the images for their ability to identify structures and support a diagnosis from the images. Simple descriptive statistics were performed.
Results
There was a statistically significant difference between the RTMUS group and the in-person group. The images captured by the in-person group were more likely to identify structures and support a diagnosis than those captured by the RTMUS group.
Conclusions
In this study, ultrasound-naive medical students who received in-person education on conducting the knee exam and FAST exam obtained higher-quality images than students who received RTMUS education. This study suggests that if RTMUS is used for medical purposes, increased ultrasound training may be needed prior to initiation of RTMUS.
Keywords
Introduction
Trauma is a common cause of morbidity in individuals who participate in outdoor activities. 1 Many of these patients may have prolonged transport times prior to definitive care. One tool that providers can use in the wilderness to help evaluate patients is portable point-of-care ultrasound (POCUS). POCUS can help providers make rapid critical clinical decisions in isolated areas. For example, participants in one study successfully used POCUS on mountains and in airplanes to diagnose pneumothoraces. 2 However, there is a wide range of POCUS experience and training among wilderness medicine providers.
Real-time remote tele-mentored ultrasound (RTMUS) can help bridge the gap between wilderness medicine providers and ultrasound experts. 3 In RTMUS, there is a novice sonographer who is connected in real time to a remote expert through audio and video communication. In real time, the remote expert can see the ultrasound movements and image feed and can guide the novice sonographer to obtain adequate POCUS images. The remote expert can then help with interpretation of these live images. Therefore, there is no delay between image acquisition and interpretation. This can be crucial for patients who have time-sensitive diagnoses and may need early intervention.
In addition to helping novices obtain POCUS views, there is a potential that RTMUS also can be used for remote instruction in POCUS. RTMUS can make POCUS more accessible to medical students who may be rotating on remote or outdoor electives as well as electives that do not provide POCUS training. 4 However, data are scarce on the effectiveness of using RTMUS to teach medical students compared with live instruction. One way to evaluate the teaching effectiveness is by performing qualitative analysis of ultrasound images.
The goal of this study was to determine the feasibility of using RTMUS to teach medical students, who are beginners in ultrasound, the Focused Assessment with Sonography in Trauma (FAST) exam and the knee POCUS exam during a wilderness medicine outdoor activity by evaluating image quality and the ability to make a diagnosis from the ultrasound images obtained.
Methods
Participant Selection
This study was approved by the institutional review boards at all participating sites. Participants were US medical students, first through fourth year, recruited from 2 large public medical schools without a POCUS longitudinal curriculum. At the first institution, students were recruited in conjunction with the wilderness medicine interest group, whose members received an email about a 1-d event involving outdoor activities and POCUS learning. Medical students from this institution were predominantly first- and second-year students. Prior to the event, the students from the first institution did not have specific hands-on training on the FAST exam or the POCUS knee exam. A total of 19 students from the first institution were consented and participated.
The second institution recruited students through its wilderness medicine elective; this group was composed of third- and fourth-year medical students. The students from the second institution did not have a POCUS elective rotation but did have more exposure to POCUS than those from the first institution through clinical rotations such as the emergency department. A total of 14 students from the second institution were consented and participated.
Design
Both institutions held a similar event on different dates. Participants gathered at a local state park on the day of the event and were randomly assigned to 2 groups without regard to medical school training year. The first group received in-person POCUS guidance for the FAST exam and RTMUS for the knee POCUS exam. Meanwhile, the second group experienced the reverse—RTMUS for the FAST exam and in-person POCUS guidance for the knee POCUS exam.
During the event, students rotated through four 20-min stations with equal instructor contact time: 2 ultrasound stations and 2 wilderness medicine activities, completing the ultrasound activities at alternating stations. The in-person POCUS stations were led by an emergency medicine ultrasound faculty member with specialized training in POCUS, accompanied by an emergency medicine resident, who provided hands-on instruction for the exams. The RTMUS POCUS station had an emergency medicine ultrasound faculty member instructing students through real-time videoconferencing from a nearby site, using teleguidance on the Butterfly Network application with a WiFi-enabled Apple iPad. This provides a split-screen view in real time of the person performing the exam and the ultrasound image feed (Figure 1). An in-person emergency medicine faculty member with specialized training in POCUS was present to handle the iPad and assist with image saving and technical issues.
Example of the Butterfly telemedicine application. Left side Ultrasound live feed. Right side live video camera. Right lower corner ultrasound expert video feed.
At the beginning of the station time, the students observed a similar demonstration of the designated station exam (at the RTMUS station, the in-person faculty member demonstrated the station exam). This went over the views for the exam and what images needed to be collected for the study. Afterwards, the students each had a chance to perform the exam on a model with faculty guidance. The hands-on portion of the education is where the groups differed. At the RTMUS station, the remote faculty instructor guided the student through the exam over videoconferencing. At the in-person station, faculty members were able to instruct students with limited touching of the actual probe in the student's hand. Each student had ∼5 min for each scan depending on the skill level of the group. The students were asked to save images while they performed the exam. Although during the allotted time per station the students performed a full exam, due to time constraints, participants saved only 1 image at the FAST exam station and 1 image at the knee POCUS exam station. At the FAST exam station, participants saved a right upper quadrant (RUQ) view, and at the knee POCUS exam station, participants saved a suprapatellar (SP) knee view. The knee exam was chosen for the musculoskeletal exam because this ultrasound exam is straightforward and easy for learners to grasp compared with other complex musculoskeletal ultrasound exams.
Image Assessment and Statistical Analysis
Images of the RUQ view from the FAST exam and the SP view from the knee POCUS exam were used for qualitative assessment. One image per student from each station type was assessed for the ability to identify structures and the ability to make a diagnosis from the image, components of the quality-assurance grading rubric recommended by the American College of Emergency Physicians. In the RUQ view, graders were looking for the ability to see the space between the liver and the kidney. In the knee view, graders were looking for the ability to see the suprapatellar bursa and the quadriceps tendon attaching to the patella. Images obtained through RTMUS and live instruction were placed in a randomized order with a grading sheet and sent to reviewers. Reviewers were blinded to the order in which the images were obtained and under which guidance. The images were graded by 2 physician graders, 1 from each institution. The image graders were ultrasound faculty who were not instructors during the session day with previous fellowship training and who regularly perform quality assurance at their respective institutions. The numbers of student-obtained RUQ ultrasound images and SP images able to identify structures and support a diagnosis were calculated separately. Kappa values between the 2 reviewers were calculated for each view and each question to assess for overall agreement in image grading.
The ability to identify structures using each view along with the ability to make a diagnosis using the view was compared between recipients of in-person and RTMUS training. Statistical analysis using comparison of 2 proportions assuming a null hypothesis of no difference was performed.
Results
A total of 33 medical students from first through fourth years participated in the study from 2 different medical schools. A varied number of images were graded for each category due to students coming in and out of the events at different times. Thus, not all students completed all the images. A total of 17 FAST and 12 knee exams were graded for in-person instruction, and 13 FAST and 16 knee exams were graded for RTMUS instruction.
The proportion of acceptable images, as determined by the 2 graders, was calculated for each image type and modality (Table 1). There was a statistically significantly higher proportion of images meeting the ability to identify structures when comparing in-person instruction with RTMUS for the knee examination (
Proportion of images graded as having the ability to identify structures.
Proportion of images graded as having the ability to make a diagnosis.
Proportion of images in summation per modality graded as having the ability to identify structures or the ability to make a diagnosis.
Proportion meeting criteria for ability to identify structures and ability to make a diagnosis along with number of images for each category.
FAST, focused assessment with sonography for trauma; RTMUS, remote tele-mentored ultrasound
The graders’ interrater reliability was assessed using Cohen's kappa test for agreement (Table 2). There was at least moderate agreement for all sets of images except the image quality to make a diagnosis for the RTMUS FAST exam, which had only fair agreement.
Cohen’s kappa values for each set of images graded by 2 reviewers.
FAST, focused assessment with sonography for trauma; RTMUS, remote tele-mentored ultrasound
Discussion
In prior studies, RTMUS has been shown to help guide individuals through ultrasound examinations using audio and video communication. 3 Nevertheless, there remains uncertainty regarding whether RTMUS can serve as a viable educational resource. The results from our study demonstrate that medical students with in-person ultrasound training were able to capture a greater proportion of images with identifiable structures and diagnostically high-quality images than their RTMUS-trained counterparts. Reasons for this difference may be that in-person training allows for better visual examples and hands-on training than RTMUS. In-person training may allow the instructor to demonstrate, in three dimensions, the motions required to obtain a better image than RTMUS. The instructors were asked not to touch the student or probe but to use other objects or their hands to mimic changes needed to improve positioning or image captured to prevent images obtained by the instructor as opposed to the student from being incorporated in the study. The RTMUS instructors also may have had difficulty communicating in language specific to ultrasound practice because the students had not had formal training in ultrasound.
The one instance where there was not a statistical difference between the groups in obtaining images of sufficient quality to identify structures was on the RUQ image. Most medical schools have incorporated some kind of ultrasound curriculum within the last several years.5–7 Although both the medical schools did not have a formal POCUS longitudinal curriculum, the FAST exam is used commonly as an introductory exam. Medical students may have had more familiarity with this type of exam prior to the session, leading to increased comfort and capability when obtaining FAST images. 8
The interrater reliability was moderate overall for most of the images reviewed. However, one area of notably lower kappa values was within the ability to make a diagnosis for the RTMUS group for the FAST exam. It may have been that 1 grader had more experience interpreting FAST exams than the other grader or felt more comfortable reaching a diagnosis based on a more limited view. Additionally, in review of the images, 1 grader could have used visualizing Morrison's pouch, the liver tip, and lower right diaphragm/lung as criteria for an accurate image, whereas the other grader may have evaluated only for visualization of Morrison's pouch. Graders were ultrasound faculty with prior ultrasound fellowship training who were regularly participating in quality assurance at their respective institutions. Formal instructions were not given to graders but anticipated using their prior experience in image review to guide their grading. Thus, potentially more or fewer of the RTMUS images of the FAST exam could have been rated as diagnostic depending on the grader and their prior experience. Finally, only still images were able to be saved as a part of this study, limiting the ability to interrogate multiple areas within the RUQ that may have improved diagnostic accuracy. Therefore, with altered or standardized criteria, the noted differences between the FAST images obtained after RTMUS and after in-person instruction could disappear.
Other smaller studies also have looked at the comparison of an in-person expert to a remote expert teaching and found no difference in success rates. One study looked at the ability of participants with no prior training to capture images of the appendix. 9 In contrast to our study, this study found no difference between remote and on-site experts. However, in this study, participants spent 5 min identifying a single structure, whereas 5 min is the amount of time recommended to complete an entire FAST examination. Thus, participants were allotted more time than usual for image capture, which could affect the results of our study. Another study evaluated traditional in-person courses as compared with tele-ultrasound POCUS courses for continuing medical education during the COVID pandemic. 10 They found that the tele-ultrasound course was as effective for improving POCUS knowledge, but the authors did not specifically look at image quality.
The RTMUS participants in our study were able to capture images that were of appropriate quality most of the time. This study further supports the potential utility of RTMUS in settings where no ultrasound experts are locally available. Examples include remote locations such as evaluation of a critically ill patient at a national park, battlefield, or rural medical site. Further studies with more ultrasound beginner participants in a variety of remote locations should be conducted. Participants in these studies likely would benefit and obtain a higher proportion of high-quality images if provided with further education prior to performing an ultrasound exam. It is likely that this was a large limiting factor in our study participants’ ability to obtain a higher proportion of high-quality images. However, it is still unclear how much and what type of prior training are required to equip ultrasound-naive or potentially medically naive students or health workers with the ability to obtain diagnostically useful images. Therefore, future studies should evaluate the most effective method of performing RTMUS.
Limitations
One major limitation of our study is that the number of images obtained for review was limited. The groups also had different numbers of participants because some students showed up late or had to leave early. Another limitation was that participants performed only 2 exams: the FAST exam and the knee ultrasound. If a different ultrasound exam were chosen (possibly a simpler exam, such as thoracic) to perform a similar study, the results between the 2 groups might be more similar. Another limitation was that all ultrasound imaging was performed on healthy volunteers, so there was no abnormal pathology to identify. Lastly, only 1 view of the FAST exam was reviewed for this study.
Finally, the small number of participants may have affected the results. Specific information regarding prior ultrasound exposure and scanning experience was not collected and correlated with the images to prevent identification and maintain participant confidentiality. Future studies could include only specific subsets of students in medical school to identify prior experience impact.
Conclusions
The quality of images obtained after RTMUS teaching of medical students was inferior to that of images obtained after in-person teaching. This suggests that further in-person ultrasound training may be needed prior to initiation of RTMUS for medical evaluation, and if used for education, learners may need additional practice to obtain similar image quality as compared with in-person teaching.
Footnotes
Acknowledgments
The authors thank Eileen O’Brien for her assistance with technical writing, editing, and manuscript preparation. Additionally, we thank the University of Maryland School of Medicine Wilderness Medicine Interest Group and the Carilion Clinic Wilderness Medicine Course for hosting the educational events that allowed for the study to be completed.
Author Contribution(s)
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: AS published the textbook,
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
