Unlocking the Potential of Augmented Reality: Investigating its Impact on Laparoscopic Training

Abstract

Laparoscopic surgery is an important procedure that is performed over 15 million times annually. However, complications can reach up to 39%. Simulation-based training has been shown to be effective in laparoscopic surgery. Specifically, Augmented Reality (AR) simulators have been shown to improve performance. Leveraging the effectiveness of AR in medical training, this two-phase study aims to explore the potential utility of AR for laparoscopic training. Results from semi-structured interviews with experts and residents revealed that the advantages of current simulators include haptic feedback and disadvantages include lack of realism and visualizations. Results also showed that the most common challenges experienced by residents include camera navigation and trocar insertion. Results from user testing show that the AR simulator was perceived to have the ability to improve trocar insertion and laparoscopic surgery skills and practices more than current simulators. Thus, the AR simulator may be beneficial in improving skills in laparoscopic surgery.

Introduction

Laparoscopic surgery is considered the gold standard for most surgical procedures (Athanasiadis et al., 2021), with over 15 million patients undergoing this procedure annually in the United States (Laparoscopy - North Kansas City Hospital, North Kansas City, MO, 2024). Laparoscopic surgeries allow surgeons to perform minimally invasive abdominal surgeries without the need for open procedures, resulting in reduced pain, recovery time, and hospitalization stay (Ancona et al., 1995; Garry, 2006). However, it is still associated with complications reaching up to 39.8% (Krishnakumar & Tambe, 2009). Studies indicate that complication rates can be reduced as surgeon experiences increase (Hobbs et al., 2006). Thus, research has significantly focused on improving laparoscopic training (Hobbs et al., 2006).

The current standard for laparoscopic training, as outlined by the American College of Surgeons and SAGES, is the fundamentals of laparoscopic surgery (FLS) (Schmiederer et al., 2022). FLS trains and evaluates knowledge and skills to perform basic laparoscopic surgical procedures, including peg transfer, knot-tying, and suturing (FLS Manual Skills Written Instructions and Performance Guidelines, 2014). Typically, FLS training involves trainees completing didactic lectures, hands-on simulation training, and evaluation exams (Valanci & Fried, 2022). While FLS training can enhance performance in the operating room (OR) (Higgins et al., 2023), research suggests a need for improving current FLS training (Athanasiadis et al., 2021). Specifically, the skills needed to perform in the OR are more advanced than the skills acquired during training (Athanasiadis et al., 2021; Fonseca et al., 2014). In addition, opportunities to improve and practice skills after achieving proficiency in FLS training are limited (Athanasiadis et al., 2021). Lastly, reaching proficiency has a high learning curve of up to 200 laparoscopic procedures performed in the OR (Kumar & Gill, 2006).

One method of improving training and reducing the learning curve is through mixed reality technologies (Hu et al., 2019). Mixed reality includes virtual reality (VR), which is a fully immersed experience, and augmented reality (AR), which integrates virtual and interactive objects into the real-world environment (Berryman, 2012). Although VR simulators can improve training, many experts and residents prefer non-VR settings where they can manipulate the actual physical instruments and objects (Botden et al., 2007).

Given the potential of AR to integrate virtual objects into actual objects (Berryman, 2012) and its improvements in teaching and learning outcomes (Dhar et al., 2021), the use of AR has been increasing in medical training (Dhar et al., 2021). AR can be effective in reducing the learning curve, improving performance and patient safety, and achieving improved surgical results (Halic et al., 2010; Linte et al., 2013). Specifically, prior work has also shown that using AR for surgical medical training can improve theoretical knowledge and practical skills (Dhar et al., 2021).

Leveraging the effectiveness of AR in medical training, this two-phase study aims to explore the potential utility of AR for laparoscopic training. The first phase of this study explores the advantages and disadvantages of current laparoscopic training devices, and the common challenges experienced by medical residents. With these results, we designed an initial prototype of an AR training simulator. Then, the second phase of this study focuses on exploring the users’ perceptions of the AR simulator and its potential impact in laparoscopic training.

Phase 1: User Needs Gathering

The purpose of the first phase of the study was to gain an understanding of the advantages and disadvantages of laparoscopic training and the most common medical challenges experienced by residents in laparoscopic surgery. Specifically, we aimed to answer the following research questions (RQ):

• RQ1: What are the advantages and disadvantages of laparoscopic training devices?

• RQ2: What are the most common challenges experienced by medical residents in laparoscopy?

Participants and Procedures

To answer our research questions, we recruited 7 experts and 3 medical residents all with experience in laparoscopic surgery at Hershey Medical Center (HMC). Experts were defined as someone who had >2 years of laparoscopic experience post-training and identified themselves as experts. While residents were defined as someone who had no or little experience in laparoscopic surgery post-training. The medical specialty of experts were 2 acute care surgeons, 1 pediatric surgeon, 3 fellows in Minimally Invasive Surgery, and 1 attending surgeon. All residents were in general surgery programs. Experts had 2 to 23 years of laparoscopic experience. While medical residents had 0 to 1.5 years of laparoscopic experience. Furthermore, all participants had received prior training in laparoscopic surgery, either through FLS or during their residency program. Demographics reported indicated a total of 7 males (6 experts and 1 resident) and 3 females (2 experts and 1 resident).

Experts and medical residents were recruited through word of mouth. Participants in this study, were first asked to read the purpose and procedures of the study and informed consent was obtained. Then participants were compensated with $10 cash. Then, 30-min in-person semi-structured interviews were conducted individually with each participant, and audio was recorded. Participants were asked around seven questions which included what main benefits/deficits of current training methods, the most common challenges and procedures experienced by residents in laparoscopic surgery, and whether the skills developed in existing training methods transfer to the clinical environment. At the end of the study, all participants were thanked for their time and participation in our study.

Qualitative Data Analysis

An inductive content analysis (Hsieh & Shannon, 2005) was conducted to understand the advantages and disadvantages of current training and common medical challenges in laparoscopic. The audio recording captured 10 interviews with a total of around 265 min. The interviews were transcribed using Otter.ai and were checked for accuracy by listening to all the interviews and editing the transcript. A codebook was then developed using an inductive analysis approach (Hsieh & Shannon, 2005). The codebook was created by two raters and included themes that were discussed during the interviews, see here for codebook. Next, each rater individually coded two interviews using NVivo. Then, the inter-rater reliability (IRR) was calculated. The IRR between the two raters was fair and acceptable (Cohen’s Kappa = 0.72). Thus, the rater coded the remaining interviews individually.

Results

This section highlights our results in terms of the research questions for the first phase of this study. For purposes of this analysis, “f” refers to the frequency that a node has been discussed during an interview, “E” is considered an expert and “R” is considered a medical resident. The codebook (here) includes definitions of all themes discussed in this section.

RQ1: What Are the Advantages and Disadvantages of Laparoscopic Training Devices?

The first research question was developed to gain a deeper understanding of the advantages and disadvantages of current laparoscopic training devices. To answer this question, an inductive content analysis showed that the box trainers’ advantages (f = 30) and disadvantages (f = 23) were discussed more frequently than the VR simulators’ advantages (f = 6) and disadvantages (f = 9). For advantages of box trainers, realism was mentioned the most frequently (f = 19) followed by haptics (f = 5) and availability (f = 3). For example, for realism, E2 stated “the instruments you use are the same instruments in the OR.” For haptics, R2 stated “You also get really nice haptic feedback as well, which is a lot of what doing laparoscopic surgery is about is feeling the tissue and feeling comfortable with tissues and dissection” For availability, E5 stated “..because everybody has to pass it, forces your program to make sure that you have e resources and the time to practice as well.” For disadvantages of box trainers, realism was mentioned most frequently (f = 16) followed by visuals (f = 2). For example, E1 stated “but it is limited because the setup is often ideal, right? Your cameras focus perfectly on the task at hand.” For visuals, E7 stated “..visualizations and the appearance is going to be different.”

For advantages of VR simulators, visuals were mentioned most frequently (f = 4) followed by realism (f = 2). For example, for visuals E4 stated “the visuals were okay, and in those you could actually do procedures” and for realism, R2 stated “[. . .] you can do the dissection of an appendectomy, take down the appendix, everything, and I feel like that’s really useful for preparing yourself for that specific procedure.” For disadvantages of VR simulators, haptics was mentioned most frequently (f = 7) followed by realism (f = 2). For example, for haptics, E2 stated “haptics in the VR trainers, the ones that I’ve used, the feel is just bizarre. I mean, some of them are better than others, but putting instruments in, you just don’t get the same feel. . .” For realism, R2 stated “..the main thing that’s the deterrent is basically the jerkiness and the lack of fluidity, it just lags out on you or your laparoscopic trainer just starts lagging out and starts giving you this weird feedback and jerks.” These results indicate that while current laparoscopic training simulators have advantages and disadvantages, simulators that integrate haptic feedback, instrument realism, visualizations, and feedback are needed.

RQ2: What Are the Most Common Challenges Experienced by Medical Residents in Laparoscopy?

Our second research question aimed to understand the challenges residents experience during laparoscopic surgery. To answer this, an inductive content analysis showed that camera/instrument manipulation (f = 19) was the most common challenge as expert 1 revealed, “I’ve definitely seen some of our juniors who need to practice the camera driving.”

The next most common challenge was trocar placement (f = 14), as mentioned by expert 4 “Trying to understand the pathology in terms of where you where you place those trocars”

Then traction/petting the tissue (f = 8), as mentioned by expert 6, “Either they’re petting the tissue, and they’re not really making progress, or they’re tearing the tissue,” and resident 2 “It’s kind of tough to figure out exactly how much traction there’s needed.”

Self-positioning (f = 7) was another common challenge as expressed by resident 2, “[. . .] trying to find that ideal position is not easy, takes a little practice and a little bit knowledge. . .”

Then patient anatomy (f = 5) was also mentioned as expert 5 expressed “[. . .] the problem is in the variability where they need to go, depending on specific areas in the body [. . .]”

Lastly, procedure steps (f = 2) as mentioned by resident 1 “[. . .] knowing the progression of the operation, and where to cut, or like what to do in a specific point in the operation to allow it to progress to the next step [. . .].”

These results indicate that medical residents experience challenges related to inserting and manipulating the camera and instruments and inserting the trocar during laparoscopic surgery. As such, there is a need to design simulators that reduce these challenges.

Phase 2: Prototype Development and Evaluation

Based on the results of the first phase, we designed an AR simulator using the Microsoft HoloLens 2. This AR simulator combines the advantages of box trainers and VR simulators and address some of the challenges identified. Specifically, we aimed to develop an AR simulator that can provide haptic feedback, improved visualizations, and feedback. The Unity software version 2021.3.6 and Mixed Reality Toolkit Foundation v1.0.2209.0 were used to design the virtual objects which included a patient, beeping heart rate monitor, and a movable operating lamp, see Figure 1.

Figure 1.

The virtual components in the AR simulator which include patient, heart rate monitor, and movable operating lamp.

Material and Methods

After developing our first prototype of the AR laparoscopic training device, the second phase of this study focuses on exploring users’ perceptions of the AR simulator. Specifically, we sought to answer the following research questions (RQ):

• RQ1: How satisfied are users with the current laparoscopic training simulators, methods, and practices?

• RQ2: How satisfied are users with the new AR simulator compared to current laparoscopic training simulators?

• RQ3: What is the overall opinion of the new AR laparoscopic training simulator?

Participants and Procedure

An exploratory study was conducted with 7 medical residents and 4 medical students from HMC. Residents were in the general surgery residency program, ranging from Post-Graduate Year 1 (PGY-1) to Post-Graduate Year 5 (PGY-5). All residents had prior experience in performing laparoscopic surgeries: 2 residents had performed 10, 1 had performed 20, 1 had performed 40, 1 had performed over 200, and 1 had performed over 800. While medical students had no prior experience in performing laparoscopic surgeries. The study included 7 male participants and 4 females.

Participants were recruited through word of mouth. Participants were first asked to read the purpose and procedures of the study and informed consent was obtained. Then participants were compensated with $15 cash. Participants were then asked to complete a 15-item pre-demo survey where they rated their response on a 5-point Likert scale ranging from “Strongly disagree” (1) to “Neither agree nor disagree” (3) to “Strongly agree” (5). Specifically, the pre-demo survey included statements related to current laparoscopic training methods and practices, see Table 1 for all statements.

Table 1.

Results of the Pre-Demo Survey.

Survey statements	Median	Z	p-value
Current laparoscopic training methods adequately prepare residents to perform laparoscopic surgeries in a clinical setting	4	1.61	.107
Current laparoscopic training methods help residents acquire an understanding of the amount of force needed while inserting trocar	3	−.54	.589
Current laparoscopic training methods help residents acquire an understanding of how to handle complications during the procedure	3	0	1
Current laparoscopic training methods recreate the psychomotor skills (trocar insertion, camera manipulation, extraction, etc.) required for laparoscopic surgery	3	−.71	.480
Current laparoscopic training methods effectively replicate the challenges of performing the procedure in a clinical setting	3	−.33	.739
Current laparoscopic training methods closely simulate the experience of performing the procedure on an actual patient	3	−1.86	.063
Current laparoscopic training methods allow for realistic feedback on performance during the procedure	3	1	.317
Resident use of a simulator to practice laparoscopic techniques improves patient outcomes	4	2.60	.009
Simulators are a beneficial supplement to conventional “hands-on” training	5	3.13	.002
Simulation training is useful for medical residents learning laparoscopic surgery	5	3.21	.001
Medical residents should achieve a predefined degree of proficiency on a laparoscopic simulator before being allowed to work on patients	4	2.50	.013
Resident use of a simulator will improve laparoscopic surgery techniques	5	2.83	.005
If made available, you would encourage residents to practice on a laparoscopic simulator	5	3.02	.003
If made available, you would mandate residents to practice on a laparoscopic simulator	5	2.52	.012
Simulation training should be an integral part of laparoscopic academic residency programs	5	2.92	.004

Participants were provided with an overview of the virtual HoloLens features, followed by instructions to wear and calibrate the HoloLens. While wearing the HoloLens, participants were first encouraged to interact with a commercially available Medicinology and Co box trainer for FLS laparoscopic skills. Then, participants were asked to manipulate the graspers and perform a peg transfer task. The peg transfer task involves using the graspers to move each peg individually from their non-dominant hand to their dominant hand midair (FLS Manual Skills Written Instructions and Performance Guidelines, 2014). After performing the peg transfer task, participants were asked to complete a 12-item post-demo survey where they rated their response on a 5-point Likert scale ranging from “Strongly disagree” (1) to “Neither agree nor disagree” (3) to “Strongly agree” (5). The post-demo survey included statements comparing the AR laparoscopic training prototype to current laparoscopic training simulators, see Table 2 for all statements. Participants were then verbally asked “what was your overall opinion of this new laparoscopic training device?” which was audio recorded. At the end of the study, all participants were thanked for their participation.

Table 2.

Results of the Post-Demo Survey.

Survey statements	Median	Z	p-value
This simulator would be more useful for training trocar insertion	3	1.73	.084
This simulator would be more useful for simulating conditions similar to that in a clinical setting compared to current training methods	4	2.89	.004
This simulator would be more useful for acquiring an understanding of laparoscopic surgery techniques and practices	4	2.88	.004
This simulator has the potential to provide realistic haptic feedback and visualizations compared to current training methods	4	2.92	.004
This simulator has the potential to improve psychomotor skills (trocar insertion, suturing, knot-tying, etc.) required for laparoscopic surgery more than current laparoscopic training methods	4	2.74	.006
This simulator has the potential to improve patient outcomes more than current laparoscopic training methods	4	2.59	.010
This simulator would be a more beneficial supplement to conventional “hands-on” training than current laparoscopic training methods	4	2.88	.004
Using this simulator would be more useful for medical residents learning laparoscopic surgery than current laparoscopic training methods	4	2.33	.020
This simulator will improve resident laparoscopic techniques better than current laparoscopic training methods	4	2.64	.008
This simulator is an improvement over current laparoscopic training methods	4	2.76	.006
If made available, you would encourage residents to practice on this laparoscopic simulator	4	2.89	.004
If made available, you would mandate residents to practice on this laparoscopic simulator	4	2.08	.037

Results

To answer our research questions, all statistics were analyzed with SPSS (v. 29.0) with a significance level of .05.

RQ1: How Satisfied Are Users with the Current Laparoscopic Training Simulators, Methods, and Practices?

The first research question was developed to understand the users’ satisfaction with current laparoscopic training. To answer this question, the pre-demo survey was analyzed. A one sample Wilcoxon signed-rank test was conducted to determine whether the scores for each statement in the pre-demo survey were significantly different than the median value of 3 (neutral). The scores were approximately symmetrically distributed, as assessed by a histogram with superimposed normal curve.

Results showed that 8 out of the 15 statements were statistically significantly rated higher than the median value of 3 (p < .05). Out of the statements that were not statistically significantly different, 1 statement had a median of 4 and all other 6 statements had a median of 3. See Table 1 for more details. These results can indicate that simulation training is perceived to be beneficial for laparoscopic training.

RQ2: How Satisfied Are Users with the New AR Simulator Compared to Current Laparoscopic Training Simulators?

The second research question was developed to understand what the users’ satisfaction with the AR training simulator is when compared to current laparoscopic training simulators. To answer this question the post-demo survey was analyzed. A one sample Wilcoxon signed-rank test was conducted to determine whether the scores for each statement in the post-demo survey were significantly different than the median value of 3 (neutral). The scores were approximately symmetrically distributed, as assessed by a histogram with superimposed normal curve.

Results showed that 11 out of the 12 statements were statistically significantly rated higher than the median value of 3 (p < .05), see Table 2 for more details. Specifically, our results indicated that the AR simulator would be more useful for training psychomotor skills such as trocar insertion/placement which is a common challenge experienced by residents that we identified in phase 1. Thus, these results provide preliminary evidence that the AR simulator was perceived to be potentially more useful and beneficial for laparoscopic training than current training devices.

RQ3: What Is the Overall Opinion of the New AR Laparoscopic Training Simulator?

The last research question was designed to understand the users’ overall opinion of the new AR simulator. To answer this question, the verbal responses to the final open-ended question were analyzed. The short audio recordings were reviewed, the number of participants expressing any opinion was counted, and the exact quotes were added. Six participants had positive opinions regarding the AR training simulator, as shown below.

“I think the haptic feedback is good and being able to use it without an additional operator required because surgical residents’ tent to have challenging hours.” ~R1, “[. . .] psychomotor like getting used to getting to picking these up like I’ve already felt that I’m manipulating this better”~ R5, “[. . .] seeing patients and monitors which is synced up to your movement makes it more of a realistic experience.” ~R3

These results highlight that the AR simulator is also perceived to be realistic, can provide haptic feedback, and improve psychomotor skills. Thus, these results provide preliminary evidence that the AR simulator can be potentially beneficial for acquiring skills in laparoscopic training.

Conclusion and Future Work

Leveraging on the effectiveness of AR in medical training, this two-phase study aims to explore the potential utility of AR for laparoscopic training. Results from the first phase of the study showed that (1) the advantages of current training included box trainers’ realism of instrument manipulation and haptic feedback, and VR’s visualizations and feedback, (2) the disadvantages of current training included box trainers’ unrealistic setup of camera and instruments, and VR’s lack of haptic feedback, and (3) residents’ main challenges were camera and instrument manipulation, traction/petting the tissue, self-positioning, and trocar placement. As such, these results highlight a need to develop more realistic simulators that help residents overcome these challenges. With these results, the AR simulator was designed, and its utility was evaluated. Results from the second phase of this study showed that (1) simulation-based training is perceived to be beneficial for laparoscopic training, (2) the AR simulator was perceived to be an improvement over current laparoscopic training methods, (3) the AR simulators was perceived to be more beneficial for acquiring psychomotor skills, and (4) the AR simulator was perceived as a more realistic training. Thus, our AR simulator can be beneficial in improving skills in laparoscopic surgery training and potentially reduce the learning curve. However, future work will be conducted to further develop the AR, collect performance feedback, and test the effectiveness of the simulator by comparing it to existing methods.

While this study provides valuable knowledge that can be used to improve laparoscopic training, there are a few limitations that need to be addressed. First, the sample size of the study is small. Second, some medical students were recruited and as such, they may not have much knowledge regarding laparoscopic training methods. Third, there was no data collected on recommendations on how to further improve the AR simulator or whether it addressed the advantages and disadvantages identified in phase one will be collected. Future work will aim to recruit participants with more experience in current laparoscopic training methods to identify whether the simulator addresses the disadvantages identified and ways to further improve upon the utility of the simulator.

Medical educators can take inspiration from this work to develop AR simulators that address challenges experienced by residents and integrate realistic OR conditions. Furthermore, with the use of AR, educators can develop more comprehensive and captivating curriculums that provide students the chance to not only learn the material but experience it which has been shown to improve learning outcomes. As such, incorporating AR to bridge the gap between theoretical knowledge and practical application can create effective learning environments that may improve performance outcomes.