Telemedicine Teaching for Everyone? Comparing Responses of Medical Students and Residents with a Shared Curriculum

Abstract

Introduction:

The COVID-19 pandemic forced rapid increases in the use of telemedicine, which requires a different skillset than in-person visits. Both medical students and residents required urgent training in telemedicine to utilize the technology for patient care. This study compares evaluation outcomes using the same shared national curriculum for medical students and family medicine residents at the same institution.

Methods:

Medical students and family medicine residents at Western Michigan University Homer Stryker M.D. School of Medicine completed and evaluated the Society of Teachers of Family Medicine Telemedicine Task Force national curriculum on best practices and foundations within the realm of telemedicine. A Mann–Whitney analysis was performed on Likert scale questions.

Results:

Medical students had significantly greater knowledge, skill, or attitude acquisition from Module 1, the basics of telemedicine, than family medicine residents. There were no significant differences in knowledge, skill, or attitude acquisition in Module 2, 3, 4, or 5.

Discussion:

The results from this single-institution sample of medical students and family medicine residents using a shared telemedicine curriculum illustrate important differences from the national data. Unlike the national data, there was a significant difference in knowledge, skill, or attitude acquisition in Module 1, while there was no difference in knowledge, skill, or attitude acquisition in the remaining four modules. Our results indicate changing utility of the telemedicine modules based on previous and ongoing experience. Learners of multiple levels find utility in a shared curriculum in teaching novel concepts.

Introduction

The COVID-19 pandemic forced health care systems to evolve rapidly. Telemedicine (clinical) and telehealth (both clinical and non-clinical) were used as physicians, and their patients sought to safely participate in medical care. ¹ In a 2021 American Medical Association survey, 85% of physicians self-reported current telemedicine use. ² Studies show high patient satisfaction with telemedicine visits.^3–5 Patients have indicated a preference for continued access to telemedicine services post pandemic. ⁶

With the rapid rise in telehealth, undergraduate and graduate medical education leaders have recognized the need for training in its appropriate, effective, and efficient use. ⁷ In Association of American Medical Colleges (AAMC) surveys of medical schools in the years prior to the pandemic, less than half included telehealth in their curriculum (2015–2017 academic years). This escalated to over 80% in academic year 2020. ⁸ The Family Medicine (FM) Residency Review Committee’s 2023 update on core competencies now includes several references to the use of telehealth. FM residency practices must have telehealth modalities available, and residents must show competence in telehealth use by graduation. ⁹

Telemedicine’s necessarily rapid uptake occurred without significant consideration of clinical education needs. ¹⁰ Clinical education leaders quickly realized the need for teaching its use to residents and medical students.^7,8,11,12 Early surveys indicated the utilization of didactic sessions and clinical experience for teaching medical students about telemedicine. ¹⁰ Educational efforts were complicated by availability of minimal educational materials and limited faculty expertise.^7,8,11,12 The Society of Teachers of Family Medicine (STFM) Telemedicine Task Force developed a national curriculum focused on best practices and foundations in telemedicine. This curriculum was specifically aimed at medical students and FM residents and was based on the AAMC telehealth competencies. Additional competencies were added based on committee consensus. Importantly, the curriculum was designed to be provided asynchronously via an interactive virtual platform. ⁷ This design, therefore, differs from the previously described didactic or clinical settings for education. ¹⁰ The curriculum was piloted at medical schools and FM residencies throughout North America and noted to be broadly effective. The curriculum consisted of five modules: 1—Intro to telehealth; 2—The telehealth encounter; 3—Requirements for telehealth; 4—Access and equity in telehealth; and 5—Future of telehealth. ⁷ This study looks at a subset of the pilot evaluation outcomes for medical students and FM residents within a single institution to assess for efficacy in both groups of learners while minimizing variability in institutional differences in education, experience, and practice patterns.

Materials and Methods

The STFM Telemedicine Task Force spent 1 year developing a telehealth curriculum utilizing the AAMC telehealth competencies and development group expertise. A detailed description of the process and curriculum has been previously published. ⁷ The five modules applied Bloom’s taxonomy levels of apply, analyze, and synthesize and involve interactive exercises and case-based decision-making. ¹³ Seventeen medical schools and 17 FM residency programs piloted the curriculum. Only one institution had both the medical school and a residency program selected, Western Michigan University Homer Stryker M.D. School of Medicine (WMed). The AAFP Institutional Review Board determined this study to be exempt and not human subjects’ research.

Population studied

The single-institution sample consisted of 30 medical students and 40 residents (Table 1). Participating medical students were primarily in their third year of training, when they typically rotated through FM within this institution. Few students in the first and second years were included due to the completion of short clinical experiences in FM during this time frame. No fourth year medical students participated in the study as they work with FM in the inpatient as opposed to the outpatient setting. Residents from across the U.S. training duration of 3 years were included. Learners evaluated each module using both closed- and open-ended questions. Closed-ended questions included a 5-point Likert scale that included strongly disagree, disagree, neutral, agree, and strongly agree.

Table 1.

Participants by Year in Program

Program	Year 1	Year 2	Year 3	Total
Undergraduate	1	1	28	30
Residency	15	13	12	40

Statistical analysis

For statistical analysis, medical students at all training levels were combined into a single group and compared with a single group of FM residents at all training levels. The decision to add a single first-year and second-year medical student into the analysis group was made, given statistical analysis was unchanged with their inclusion. A Mann–Whitney analysis (α = 0.05) was conducted on both medical student and resident acquisition of knowledge, skills, or attitudes to improve clinical practice and the utility for a learner at their level of training. The analysis was conducted within each module separately. Answers of highly agree and agree as well as highly disagree and disagree were combined in the analyses. Therefore, the analyses compared the results of agree, neutral, and disagree.

Open-ended comments about each module were collected in response to the prompt: “what could STFM do to improve this module?” (Table 2). Following in-depth reviews of all comments, they were grouped into five thematic categories for analysis: no suggestions, shorten, improve interactivity, increase depth, and other. Thematic categorization was determined by one of the investigators (S.D.B.) and reviewed and agreed upon by the remaining investigators. Mann–Whitney tests (α = 0.05) were run to test the association between the comment and learner level by module.

Table 2.

Comments from Open-Ended Question—“What Could STFM Do to Improve This Module?”

Module	Student level	Comment grouping	Comments included
1	Resident	No suggestions	N/A, nothing, its fine, no suggestion, its good, no comments, great module, nothing comes to mind, idk
		Shorten
		Improve interactivity	More videos, video, more interaction, video/visuals, accompanying videos, incorporate sample/simulated video visits, make it shorter and more interactive, add video
		Increase depth	Include examples, more pictures, audio, video supplementation of good and bad interactions—how to overcome barriers, may be use more tables and illustrations
		Other	Make it faster, improve redundancy of this survey
	Medical student	No suggestions	NA, none, nothing, can’t think of anything, its well done, I like how there were accompanying videos for this module, not sure, well done
		Shorten
		Improve interactivity	More interaction would make the information more memorable, more interaction, interaction, videos
		Increase depth	More examples of appropriate uses
		Other	Summary at the end, keep updating as evidence comes in
2	Resident	No suggestions	Well done, none, nothing, N/A, it’s well designed, I like how there were accompanying videos for this module, not sure
		Shorten	Make it shorter, videos were redundant, shorten videos
		Improve interactivity	Interactive, use real-life examples, more videos, more interactive, more visualization
		Increase depth	More emphasis on physical exam, tailor to residents—some very basic info included
		Other	Summary page
	Medical student	No suggestions	N/A, nothing, none, nothing it was great
		Shorten	Probably make it a little shorter, less long videos or allowing a 2× speed function, shorten it
		Improve interactivity	More videos, interactive visit, more virtual PE ideas
		Increase depth
		Other	Provide video transcript, make it faster
3	Resident	No suggestions	N/A, none, nothing, nothing comes to mind, its good, good module, all good, no comment
		Shorten
		Improve interactivity	More tables, more visualization, more pictures
		Increase depth	Real-life examples, nothing—the module itself was good for someone unfamiliar with televisits, but as someone who has been doing them all year, this is repeat information
		Other	More troubleshooting info, more questions, the evaluation questions didn’t all correlate with the available material, good for those with little computer experience otherwise not very helpful, summary page
	Medical student	No suggestions	N/A, none, no comment, not applicable, I thought the length was good
		Shorten	Shorter
		improve interactivity	Video, more downloadable note sample statements
		Increase depth	More relevant information, make information more relevant, provide more information on peripherals how they can be used and incorporated into a telemedicine encounter
		Other	Instead of linking to the different organizations, include brief overviews of general tips providers should keep in mind for telehealth visits
4	Resident	No suggestions	Nothing, none, well done, no comment, “-,” not sure
		Shorten
		Improve interactivity	Graphics, more videos, more videos of the interactions
		Increase depth	More examples
		Other	Summary page
	Medical student	No suggestions	N/A, none, nothing, this was a really good one
		Shorten	Shorter
		Improve interactivity	Videos, simulations with actual encounters might be nice
		Increase depth	Tailor info to year of training, more material regarding crisis usage, more with special needs patients
		Other	Downloadable summary, this is a telehealth module—only include what is relevant to telehealth
5	Resident	No suggestions	N/A, none, nothing, no comment, “-,” nothing comes to mind
		Shorten
		Improve interactivity	Videos, videos/graphics, more videos, more interactions
		Increase depth	More examples, more info about different tools in telemed tool boxes
		Other	Everything, not very helpful—consider removing, not really implementable in current residency practice—could touch more on the accuracy and feasibility of implementation of some of the remote devices (how well do noises transmit with electronic/remote stethoscopes, can patients place them appropriately on chest/back? etc.), summary page
	Medical student	No suggestions	N/A, no comment, none, nothing
		Shorten	Shorter
		Improve interactivity	Videos, maybe more videos demoing the tech rather than a lot of words about it
		Increase depth	Talk about drawbacks in fast advancements, provide links/access to more resources on the topics discussed
		Other	Downloadable summary, realistic skepticism on the future of telehealth—a lot of telehealth ideas sound like sales pitches, you need to consider what would incentivize use and will the use be as intended or desired. Many ideas of what telehealth could be are intriguing but lack the necessity or incentive from both providers and patients to make them happen

Results

Comparative analyses of student and resident evaluations of the modules were significantly different in just two areas (Table 3). Student assessment of the level of knowledge gained in Module 1 was significantly greater than resident assessment (p = 0.0292). There was no significant difference in ratings on the level of knowledge gained between students and residents for the four remaining modules. There was a significant difference in student and resident open-ended comments on Module 4, with more residents than students commenting that the module was right on target. There was no significant difference in student and resident responses to categorical questions on knowledge gained and the level of the module.

Table 3.

Student and Resident Evaluation of Modules

	Module 1	Module 2	Module 3	Module 4	Module 5
I gained new knowledge, skills, and/or attitudes	MS: 83R: 60(0.029 * )	MS: 56R: 73(0.134)	MS: 58R: 61(0.771)	MS: 71R: 62(0.454)	MS: 67R: 60(0.577)
Information useful at my level of training	MS: 90R: 90(1)	MS: 84R: 90(0.88)	MS: 71R: 78(0.421)	MS: 79R: 86(0.453)	MS: 95R: 88(0.785)
Comments (no changes)	MS: 77R: 60(0.170)	MS: 68R: 68(0.987)	MS: 71R: 76(0.853)	MS: 67R: 88(0.038 * )	MS: 67R: 75(0.519)
N	70	66	65	66	61

Percentage of agreement by medical students (MS) and residents (R) by module.

*

p-value (p) significant at p ≤ 0.05.

Discussion

Medical students and residents have had limited exposure to telemedicine teaching. ¹⁴ This pilot of a shared STFM curriculum in medical students and FM residents at a single institution demonstrated a significantly greater knowledge, skill, or attitude acquisition in medical students versus FM residents in Module 1. There was no significant difference between medical student and resident knowledge, skill, or attitude acquisition in any of the other modules. This is in contrast to national data that observed significantly greater agreement with knowledge, skill, or attitude acquisition of medical students versus FM residents in Modules 2, 3, 4, and 5, but not on Module 1, where there was no difference between the two groups.

Many of the WMed FM residents participated in rapid uptake of telemedicine early in the pandemic, in advance of most medical student exposure and prior to curricular exposure. Given their first-hand experience with telemedicine, Module 1, which focuses on appropriate uses of telemedicine along with its benefits and limitations, was likely already intuitive to the resident group.

Our expectation is that over the next several years, most residents will have received teaching on the basics of telemedicine as part of their undergraduate curriculum, leading to less need for introductory education in residency. WMed FM residents received little explicit instruction in the performance of telemedicine visits (Module 2), technological requirements and legal considerations of telemedicine (Module 3), the impact of telemedicine on health care access and equity (Module 4), and the future of telemedicine (Module 5) prior to its clinical use. Therefore, within the WMed institution, Modules 2, 3, 4, and 5 represented new information to both FM residents and medical students. Additionally, WMed FM resident interest in these modules may have been positively affected by their ongoing participation in telemedicine visits with resulting need and desire to improve skills in its use. With expanded teaching and utilization of telemedicine in undergraduate education, we may see a decline in resident satisfaction of the knowledge gained. This would be more similar to the national data, though it is unclear whether national data indicate increased knowledge or a lower perception of utility in FM residents versus medical students in Modules 2–5.

Medical education is a continuum from an inexperienced learner to a master clinician. ¹⁵ This pilot suggests that in relatively novel areas of medical education, curriculum can be developed and used for multiple levels of learners. Intentional use by learners allows the opportunity for learning in a spiral manner, with participant’s ability to revisit areas of weakness. With continued use of telehealth, expansion of telemedicine education to include higher-level competencies, such as incorporation of physical exam, is needed.

While this study was limited by location within a single site, this limitation was leveraged to control for outside variables, allowing for consistency in the analysis between the two learner groups. Evaluation was limited to acquisition in knowledge, skill, and attitude, as well as the level of instruction. In future studies, we would include a faculty group as part of the learning continuum.

Knowledge of efficacy of a shared curriculum for both residents and medical students is a critical message for faculty looking to deploy innovative practices. We postulate that curriculum development can focus less on the stage of the learner and more on previous exposure.