Abstract
OBJECTIVES
Telesimulation utilizes telecommunication technology to engage learners in simulation while in different physical locations. Despite this potential advantage, understanding of the student experience and assessment of student learning in telesimulation activities is limited. This study evaluates medical student emotional experience and self-identified learning in telesimulation through the Kolb experiential learning framework and qualitative analysis.
METHODS
Fourth-year medical students enrolled in the Spencer Fox Eccles School of Medicine at the University of Utah participated in 3 telesimulation activities as part of a required internal medicine course. Students were surveyed regarding their satisfaction with the activity (N = 114) and responded to questions about their emotional experience and self-identified areas of learning. Free-text responses were analyzed using qualitative content analysis to identify themes until thematic saturation (N = 66).
RESULTS
Students were highly satisfied with telesimulation, with greater than 90% of students expressing a positive view of simulation realism, debrief quality, and group size. Themes of anxiety and uncertainty, confidence versus incompetence, team dynamics, fun, and difficult patient interaction were identified regarding the emotional experience. Themes of communication and teamwork, managing emotions, information gathering, differential diagnosis, resource reference, executing treatment, and medical knowledge were identified regarding student-identified learning.
CONCLUSION
In this analysis of medical student experiences with telesimulation, we found students have rich emotional, cognitive, and behavioral experiences and self-identify learning across a variety of domains. Our findings support further study of telesimulation for medical student learning and demonstrate how assessment of outcomes via Kolb framework, using the learner's reflective observation and self-identified learning, may help better define learning outcomes from simulation.
Introduction
Simulation is a popular modality for health professionals in training that offers benefits of a replicated clinical learning environment, realistic human behavior and interaction, and authentic emotional experience in an environment where mistakes do not result in patient harm.1,2 Simulation content can be presented in a range of formats—from low to high-fidelity simulation, skills or task-trainer based, and in-person or online. At the beginning of the COVID-19 pandemic, telesimulation, a combination of telecommunication and simulation to synchronously engage physically separated learners was adapted to provide a safe learning environment.3–9 Although currently less burdened by pandemic restrictions, medical educators remain interested in remote learning modalities including telesimulation. 10
Evaluation of learning outcomes in telesimulation suggests that many learners find it as effective and favorable as in-person simulation.4,7,11–13 Telesimulation can result in knowledge gain for complex clinical cases, and with appropriate set-up can also impart technical skills.3,14,15 However, there can be variability in learning outcomes between in-person and remotely facilitated simulation-based training, with students perceiving telesimulation as inferior in some contexts. 4 Moreover, degree of learner engagement in telesimulation is not clear, as some learners may mute themselves or have cameras off which limits faculty ability to interpret emotional response. 3 Further exploration of learning associated with the telesimulation experience is needed to understand how learners acquire knowledge and skills in this setting.
Conceptual Framework
To understand how learners navigate acquisition of knowledge and skills in the setting of telesimulation, we used Kolb's experiential learning theory as a framework. In this theory, learners must undergo an experience, consciously reflect on the experience which leads to abstracting concepts, and then finally apply those abstractions for active experimentation. 16 The theory emphasizes that learning is a process grounded in experience, and that measurement of learning should not be limited to behavioral outcomes but can encompass the learner's creation of knowledge through reflection and abstraction. 16 Applied to simulation, Kolb's theory can help illuminate (a) how learners navigate the experience, and (b) how additional learning occurs outside of set objectives. This builds on previous work describing how students take away learning points from simulation.17,18 Understanding the application of experiential learning theory in the setting of telesimulation may illuminate how learner acquisition of information differs from in-person simulation and offer areas on how the experience can be optimized for learning.
Research Questions
In this study, we explored how fourth-year medical students responded to 3 simulation cases presented via telesimulation in an internal medicine course. We aimed to evaluate students’ satisfaction with telesimulation and understand their emotional experience as well as self-identified areas of learning.
Methods
Study Context
All fourth-year students at the Spencer Fox Eccles School of Medicine at the University of Utah are required to take Advanced Internal Medicine (Adv IM), a 4-week course. Students complete 2 telesimulation sessions during the course. The first session consists of 1 telesimulation scenario followed by a facilitated debrief (30 min each). The second session consists of 2 telesimulation scenarios each followed immediately by a debrief (20 min for each simulation and debrief). All simulation scenarios involved a hospitalized patient with acute cardiopulmonary complaints (dyspnea, chest pain, palpitations) in a cross-cover setting. “Cross-cover” refers to the common practice of providing urgent care for a hospitalized patient for whom one is not the primary provider, often in an overnight setting. Scenarios included a standardized patient acting as the patient and the physician facilitator acting as the bedside nurse. Debriefing was in the form of facilitator-guided post-event debriefing without video review or other formal structure. 19 Telesimulation was conducted synchronously using Zoom videoconferencing software (Zoom Video Communications). See Table 1 for key elements of the simulation. 20
Key elements of simulation-based research.
Study Design & Participants
This project utilizes 2 data sets collected from students during and after Adv IM: survey data from end-of-course evaluations (AY20-21) and survey data from assignments completed in Canvas (AY20-21). End-of-course survey questions were close-ended, administered using Qualtrics, and included questions about student satisfaction with the course and didactic sessions. Students responded to end-of-course surveys after completing Adv IM. In AY20-21 the overall survey response rate was (97%, N = 114). Survey responses were collected without student identifiers so that they data would remain anonymous. This study was deemed exempt by the University of Utah Institutional Review Board (IRB_00109278) and informed consent was waived.
Students completed the Canvas assignments immediately after participating in the simulations. These questions included: (a) What were your initial reactions to the simulation case(s)?; (b) List 3 things you learned during the simulation case. Please be as specific as possible. (c) What are 1–2 things you will change in your practice as a result of participating in this simulation case? These surveys were required assignments and were submitted to be reviewed by the course directors (SR and JG). The completion rate for all surveys was 100%. Each student had 2 sets of responses: one after the first simulation and one after the second. Responses to each simulation session were analyzed separately.
Open and close-ended survey questions are included in Supplemental material, supplement 1. Questions had been pilot-tested on over 100 medical students in AY19-20 prior to this study with minimal alterations to reflect a change to virtual simulations.
Although responses to surveys were not anonymous because they were linked to student Canvas accounts, the data was downloaded from Canvas before being analyzed so that responses were not associated with student names during analysis. De-identified student responses are available upon request from the corresponding author.
Quantitative Data Analysis
Responses to end-of-course evaluations were analyzed using descriptive statistics in Microsoft Excel.
Qualitative Data Analysis
Free-text responses from Canvas were exported to Dedoose for coding. Using an inductive approach, one author (JG, man) coded all responses using qualitative content analysis. 21 JG was trained in content analysis by SR, who has previous experience with qualitative data analysis. After reviewing and refining the initial codes and creating a preliminary codebook, an additional author (LH, woman) used the working codebook to code responses. JG and LH are both physicians experienced in medical student education and simulation facilitation. JG and LH met to review discrepancies in coding and to further refine the codebook. Once the codebook was finalized, JG and LH reviewed the data once more to ensure that all data was well-described by the codes. Several steps were taken to ensure trustworthiness: coding was conducted by multiple authors in order to explore multiple interpretations of the data, and JG and LH engaged in peer debriefing with SR during the coding process, who also provided input on findings.
Student responses to Question 1 (“What were your initial reactions to the simulation case(s)?”) were analyzed separately and described under the overarching theme of “emotional reactions”. Student responses to Questions 2 and 3 (“List 3 things you learned during the simulation case. Please be as specific as possible.” And “What are 1–2 things you will change in your practice as a result of participating in this simulation case?”) were analyzed together due to the similarity of responses and described under the overarching theme of “learning points and practice changes”.
Each overarching theme was further subdivided into sub-themes, which are described in more detail in the results. We stopped coding new data when thematic saturation was achieved after analyzing responses from 66 consecutive students collected between July 2020 and January 2021.
Results
Quantitative Results: Student Satisfaction
Student satisfaction with simulation sessions was assessed with end of course surveys (Table 2). Greater than 90% of students expressed a positive view (“agree” or “strongly agree”) of simulation realism, debrief quality, and group size.
Participant satisfaction with telesimulation (n = 114).
Qualitative Results
Qualitative data was organized into 2 main themes: emotional reactions and learning points and practice changes.
Emotional Reactions
Findings on students’ emotional experiences were organized into the following themes: Anxiety and uncertainty, confidence versus incompetence, team dynamics, fun, difficult patient interactions. Representative quotations for each theme are shown in Table 3.
Emotional response to simulation.
Anxiety and Uncertainty
Simulations evoked a wide range of emotions generally associated with anxiety and uncertainty. Students described feeling anxious, intimidated, fearful, nervous, panicked, paralyzed, and worried. Some emotions were connected to the simulation case, including fear of making an incorrect decision or harming the patient. These emotions were also driven by factors external to the simulation case, such as pressure from performing in front of peers or confusion about suspending disbelief in a simulation setting.
Confidence versus Incompetence
Students reported that the simulation experience contributed to building confidence in areas where they encountered success and creating feelings of incompetence when experiencing failure. Confidence was accompanied by emotions of calm, focus, and accomplishment, while feelings of incompetence were accompanied by embarrassment, guilt, and inadequacy. However, students were also able to identify individual weaknesses in simulation as opportunities for future growth and skill building.
Team Dynamics
Students also identified conflicting emotions regarding their experience working with peers during the simulation. Well-functioning teams brought students a sense of confidence, reassurance, additional medical knowledge, and a source of peer-to-peer learning. However, teamwork in the simulation also led to challenges and frustration if roles were unclear, leadership was absent or overbearing, or teams faced disagreement.
Fun
Students also described the experience as fun and enjoyable. Students enjoyed the opportunity to “act” as an intern, a role which they look forward to with excitement.
Difficult Patient Interaction
Interactions with standardized patients also impacted the emotional experience of students. Students noted difficulty addressing the patient's emotional and physical comfort while working through the case, particularly in the setting of uncertainty and an evolving clinical course. Concerns about the patient's perception of the students’ performance also contributed to emotions of anxiety and inadequacy.
Learning Points and Practice Changes
Students’ self-identified learning points and future practice changes were organized into the following themes: communication and teamwork, managing emotions, information gathering, differential diagnosis, resource reference, executing treatment, and medical knowledge. Representative quotations for each theme are shown in Table 4.
Learning points and practice changes.
Communication and Teamwork
The importance of communication was mentioned on many different levels. Students described learning about the role of communication within a team in responding to an acute clinical decline, communication between physicians and nurses, between physicians and patients, between medical floor physicians and ICU physicians and consulting specialists, as well as between primary medical teams and overnight cross cover. Communication in these contexts was delivered in person, telephone, electronic health record documentation, or provider hand-off tools (“sign out sheets”).
Managing Emotions
Students learned the importance of managing their emotional response to a medical situation, particularly in countering anxiety with intentional calmness and focus. This theme was also characterized by balancing a sense of urgency and decisiveness with identifying moments for pause and data synthesis.
Information Gathering
Students identified a wide variety of sources of information important to patient care in the acute setting. These include history gathered from the patient, vital signs and a physical examination of the patient, documented course in clinical notes, use of EKGs and other diagnostic tests.
Differential Diagnosis
The diagnostic process was identified as an important point of learning and practice change. Students appreciated the importance of maintaining breadth in a differential diagnosis to mitigate anchoring on an incorrect diagnosis. They also recognized the value of having a standardized diagnostic approach (or “diagnostic schema”) for commonly encountered clinical problems to facilitate an efficient and targeted evaluation.
Resource Reference
Point-of-care information reference emerged as an important area of learning. Students recognized the importance of knowing how to rapidly access critical diagnostic and treatment algorithms at the point of care, such as advanced cardiac life support algorithms.
Executing Treatment
Students learned practical aspects of treatment plan execution. These included medication administration and required monitoring, escalating level of care for higher risk treatments, and anticipating treatment needs by ensuring adequate equipment and intravenous access was available.
Medical Knowledge
Simulation also led to learning in general medical knowledge. The diagnoses in our 3 simulated cases were acute pulmonary embolism, supraventricular tachycardia, and acute pericarditis. Students described learning relevant to the diagnoses including disease presentation, diagnosis and diagnostic scoring tools, risk stratification, and severity assessment tools. A few specific learning points mentioned by students include as follows: Well's score for pulmonary embolism, pulmonary embolism severity index, impact of risk stratification of pulmonary embolism on treatment plans, pharmacokinetics of different anticoagulants, contraindications to anticoagulation and thrombolytics, diagnostic criteria for acute pericarditis, kinetics of troponin elevation, and impact of clinical stability on treatment of arrythmia.
This included disease presentation, clinical risk prediction tools, disease severity assessment, and other clinical pearls relevant to the simulation cases.
Discussion
In this study of the student-reported experience across 3 telesimulation cases, we found that students were highly satisfied with the learning experience, describe a range of emotions working through cases, and experience rich learning that goes beyond outlined objectives. In the context of the Kolb experiential learning theory, students in this study describe telesimulation as a fertile ground for experiential learning, demonstrating the ability to reflect on a simulation experience and then identify opportunities for practice change through abstract conceptualization.
Our results suggest that the learning environment of telesimulation can evoke strong emotional reactions and convey the complex team interaction, diagnostic reasoning, and data gathering. These findings are consistent with Nomura et al who recently showed an equivalent emotional experience for medical students participating in telesimulation as compared to in-person simulation. 22 Our results also add to prior work which demonstrated high student satisfaction with telesimulation and that knowledge and skill acquisition can be similar for telesimulation compared to in-person simulation.4,7,8,13
While telesimulation may make it harder to monitor learner engagement in real-time due to limitations of video or audio equipment, the student responses in our study suggest that learners were engaged at an emotional, cognitive, and behavioral level. 3 Furthermore, the simulation environment is an important determinant of extraneous cognitive load, and reduced extraneous cognitive load in a virtual environment may even enhance learning for novices.23,24 However, recent work also indicates that videoconferencing may negatively impact conversation patterns and cooperation compared to in-person communication.25,26 Further work may investigate the cognitive load of telesimulation environments as well as impacts of videoconferencing-based communication in telesimulation (with standardized patients, within teams, and during debrief).
The Kolb experiential learning theory is a useful lens through which to view the learning experience of telesimulation. The simulated clinical encounter serves as shared concrete experience for students. This is followed by a faculty-facilitated group debrief, encouraging reflective observation. In our study, reflective observation was apparent through sub-themes of “emotional reactions” (eg, “anxiety and uncertainty”). We then prompted students to identify future changes in their clinical practice based on their reflections. These abstract conceptualizations are evident in the theme of “learning points and practice changes,” in which students generate or adapt abstract ideas regarding multiple facets of a patient encounter (eg strategies to improve team communication or utilization of point of care resources). Although this study does not assess active experimentation with these new ideas, the course provided students with real overnight clinical experience similar to the telesimulation scenarios as an opportunity to complete the experiential learning cycle.
In this study, the telesimulations were embedded in a required course in the fourth year of medical school, where students present with variable knowledge in skill in management of unstable patients and are anticipating a wide range of future post graduate training experiences. Use of the Kolb experiential learning theory to analyze student reflection and conceptualization also allowed course directors to assess learning across a variety of student skill levels and range of career goals. For example, some students self-identify learning points more specifically applicable to inpatient internal medicine practice (eg pulmonary embolism risk stratification and therapy), while others identify learning that would be broadly applicable to other career paths (eg team communication, managing emotions in stressful settings).
Experiential learning theory may also uncover important impacts of telesimulation beyond intended learning objectives. While prior work has suggested telesimulation should focus on cognitive and behavioral skills, 11 the measurement of these skills often relied on observation or test questions. Student free-text responses to emotional reaction, learning points, and anticipated practice changes covered a wide array of learning outside of the set objectives for the simulation, suggesting that analysis of this type may be helpful to define the learning from telesimulation. For example, while patient communication was not an explicit learning objective for our cases, students described emotional engagement with the patient experience and learning around physician-patient communication amidst clinical uncertainty. Future work could assess the impact of simulation on students’ patient communication as an unanticipated area of practice change.
Overall, our findings support telesimulation as a satisfying as well as cognitively and emotionally rich learning modality for medical students. Although COVID-19 restrictions are currently less impactful on student learning than during the study period, medical educators remain interested in remote learning modalities. 10 Telesimulation offers some persistent advantages over in-person simulation such as reaching students who are completing clinical rotations at distant sites. Furthermore, our study demonstrates the utility of the Kolb experiential learning theory to better understand learning outcomes from simulation activities, which in this case identified areas of learning beyond the intended objectives.
Our study is not without limitations. We did not compare telesimulation to in-person simulation and cannot draw conclusions regarding the similarity or differences between these modalities. Students were required to submit reflections to questions to receive credit for attendance; although not necessarily graded on content, students may have provided responses they felt were socially acceptable. It was a single-center study with 1 cohort of fourth-year medical students, which limits generalizability. However, the rigor of the study was increased by several techniques including utilizing multiple coders, engaging in peer debriefing, and achieving a high response rate on surveys.
Conclusions
In this analysis of student experiences with telesimulation, we found that students have high satisfaction and rich emotional, cognitive, and behavioral experiences. Our findings support further study of telesimulation for medical student learning and demonstrate how learning outcomes may be defined by using the Kolb framework to assess learner's reflections.
Supplemental Material
sj-docx-1-mde-10.1177_23821205231216067 - Supplemental material for Medical Student Experiential Learning in Telesimulation
Supplemental material, sj-docx-1-mde-10.1177_23821205231216067 for Medical Student Experiential Learning in Telesimulation by John P. Gerstenberger, Lara Hayes, Candace J. Chow and Sonja Raaum in Journal of Medical Education and Curricular Development
Supplemental Material
sj-docx-2-mde-10.1177_23821205231216067 - Supplemental material for Medical Student Experiential Learning in Telesimulation
Supplemental material, sj-docx-2-mde-10.1177_23821205231216067 for Medical Student Experiential Learning in Telesimulation by John P. Gerstenberger, Lara Hayes, Candace J. Chow and Sonja Raaum in Journal of Medical Education and Curricular Development
Footnotes
Acknowledgements
We thank the staff of the University of Utah College of Nursing Simulation Center for hosting our simulation and providing technical expertise.
DECLARATION OF CONFLICTING INTERESTS
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
FUNDING
The authors received no financial support for the research, authorship, and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
