Abstract
Background and Aims:
Simulators are widely used in occupations where practice in authentic environments would involve high human or economic risks. Surgical procedures can be simulated by increasingly complex and expensive techniques. This review gives an update on computer-based virtual reality (VR) simulators in training for laparoscopic cholecystectomies.
Materials and Methods:
From leading databases (Medline, Cochrane, Embase), randomised or controlled trials and the latest systematic reviews were systematically searched and reviewed. Twelve randomised trials involving simulators were identified and analysed, as well as four controlled studies. Furthermore, seven studies comparing black boxes and simulators were included.
Results:
The results indicated any kind of simulator training (black box, VR) to be beneficial at novice level. After VR training, novice surgeons seemed to be able to perform their first live cholecystectomies with fewer errors, and in one trial the positive effect remained during the first ten cholecystectomies. No clinical follow-up data were found. Optimal learning requires skills training to be conducted as part of a systematic training program. No data on the cost-benefit of simulators were found, the price of a VR simulator begins at EUR 60 000.
Conclusions:
Theoretical background to learning and limited research data support the use of simulators in the early phases of surgical training. The cost of buying and using simulators is justified if the risk of injuries and complications to patients can be reduced. Developing surgical skills requires repeated training. In order to achieve optimal learning a validated training program is needed.
Keywords
INTRODUCTION
Laparoscopic techniques are being considered as standard methods in cholecystectomy, bariatric and anti-reflux surgery, as well as in gynaecology, amongst others. During 2008, altogether 7800 cholecystectomies were performed in Finland, of which 85% were laparoscopic (1). Bile duct injury is the most commonly reported severe complication during cholecystectomy. Its incidence is higher (0.4–0.7% vs. 0.2%) with laparoscopic technique than in open surgery (2 –6).
The specific problems of video assisted surgery are mainly related to getting used to the instrumentation and the ability to handle picture, space, and movement in a two dimensional image (7). The instruments are long and their paradoxical movements ergonomically restricted. These challenges have recently been reviewed in this journal (8).
Basic surgical training is insufficient for the learning of laparoscopic surgery. Therefore, both training and teaching environments are being improved (9, 10). Aircraft pilots have been trained with simulators since the beginning of the II World War and information technology has been used in flight simulators since the 1980's. In aviation, research results have shown that simulator training significantly shortens the time to reach the qualification (11). Simulation is also used in other high-risk fields such as nuclear power plants and military function (12). The use of surgical simulators while practising different procedures aims at risk reduction on an ethically sound basis.
Different training models (phantoms) have been used in basic medical training outside clinical work; e.g. in anaesthesiology. In surgery, procedure training was started with animal models and black box training. The newest available methods are computerised virtual reality (VR) simulators.
Simulation in surgery can be defined as learning methods or strategies, in which a learning device or exercises are created to reproduce or represent conditions that are similar to or likely to occur in actual performance (13). The most elementary first generation simulator is a surgical black box. It is a closed box with an inside camera which transmits to a monitor the image of three dimensional tasks being performed with laparoscopic instruments. New-generation computer-based simulators use VR technology with three dimensional graphics. Surgical tasks and exercises are presented on a computer screen and the instrument tips are represented on a screen as virtual images. In the latest software versions of VR simulation (e.g. LapSim and LapMentor), anatomical structures can be simulated in addition to graphical games. These latest equipment can be used to exercise complete operations or partial tasks from procedures. The tissues are simulated as realistic as possible to enable e.g. bleeding and perforations. Additionally, the haptic sensation is available in the instrument ports as a robotic force feedback element. The performance data are also captured in user logs as various parameters, which can be used to monitor training progress, as well as to set standards and goals to skills training and learning.
The hindrances of implementing simulators to surgical training are high purchasing cost and challenges related to integration of simulator training to clinical practice and surgical curriculums. (14). Specific criteria have been set up for effective simulation training. Simulator training programs should be validated, objectively assessed, and mandatory for residents. The training should be realised, distributed, and based on proficiency, and further learning should be continued in the real authentic situations as soon as possible after simulator training period (15).
There are several technically advanced and expensive simulators on the market, which are thought to make possible versatile surgical training. Laparoscopic cholecystectomy is the most advanced simulator procedure. The cost of VR simulators varies between 60 000 and 100 000 Euros. Program modules for additional exercises or operations (such as intestinal resection, gastric by-pass, hernioplasties or adnex surgery) each will add the cost by 17 000 Euros.
This systematic review aims to provide an update on computer-based virtual reality (VR) simulators in training for laparoscopic cholecystectomies. Furthermore, skills learning with VR simulators is compared to learning on black box trainers.
MATERIAL AND METHODS
Together with the National Institute for Health and Welfare, the Finnish hospital districts have established a systematic appraisal of new technology, Managed Uptake of Medical Methods program (MUMM, HALO-ohjelma), to identify and evaluate new technologies in the specialised health care. The aim of the program is to assess evidence of safety and efficacy/effectiveness of new technologies to support the decision making among hospital districts concerning the uptake of these technologies. The council of MUMM program gives recommendations about the application of technologies among hospital districts by using traffic light signals. In 2009, laparoscopic simulator training was approved by hospital districts for a systematic literature review and assessment within the MUMM program. The initial review was published in Finnish (16). The review was focussed on the transferability of simulator trained skills to live laparoscopic cholecystectomies.
Studies reporting the use of the newest generation of cholecystectomy simulators in laparoscopic training were systematically reviewed by using the following PICO setting: P population of interest, I intervention, C comparator (or control intervention), O outcome.
P: a surgical trainee learning laparoscopic cholecystectomy
I: learning by a virtual reality simulator
C: patient-based training in the operation theatre, or other forms of technical training outside the operating theatre (e.g. black box training)
O1: clinical outcomes in surgical patients: mortality, complications, conversion to laparotomy
O2: technical measures: scores of technical skills, use of materials, operation time, etc.
O3: learning indicators: rating of learning results, error rates, autonomy, decision making, etc.
SYSTEMATIC REVIEW
The initial literature search was performed from Medline-, Cochrane-, Embase- and HTA-databases in October 2009 and updated in February 2011. The search terms included virtual reality and laparoscopic cholecystectomy among others, and searches were conducted without language restrictions. Time limitation for the literature search was five years. The ongoing trials were searched from the following registries: Clinical Trials.gov and metaRegister of Controlled Trials. The search strategies will be provided by the authors upon request. Based on pre-set PICO criteria, altogether three reviews / HTA-reports and 17 original studies were identified, of which full text articles were ordered. The inclusion criteria were as follows: randomised or comparative studies about (virtual reality) simulation with at least five (in randomised) or ten (non-randomised comparative) study objects per group. Fourteen original studies met the inclusion criteria. These studies were completed by two older studies (three articles) recognised by one of the systematic reviews (17). Three registered ongoing trials were found. A completion search concerning comparison of back boxes and simulators was performed in January 2010. Altogether 13 original studies were identified, of which seven full text articles were included.
VALIDITY OF ORIGINAL STUDIES
For the assessment of methodological validity of original articles, the quality criteria from the Australian Safety & Efficacy Register of New Interventional Procedures — Surgical (ASERNIP-S) report were used (17). Only randomised studies were assessed. Requirement for adequate quality was fulfilled with five yes -answers including blinded assessment (question 2). Not reported or unclear information were considered as negative. The questions were as follows:
Was the method of randomisation adequate?
Was the outcome assessor blinded to the intervention?
Was the analysis by intention-to-treat?
Was the power calculation done?
Was the drop-out rate described and acceptable?
Was the timing of the outcome assessment in all groups similar?
Were the assessment tools validated?
Were the inclusion criteria clearly described?
Were the groups similar at baseline regarding the most important prognostic indicators?
RESULTS
SIMULATOR TRAINING
The health technology assessment (HTA) report of the ASERNIP-S from 2007 was accepted as a background document and its results dealing with laparoscopic cholecystectomies were incorporated into our review (17). In the ASERNIP-S report, study questions were formulated to describe how the skills transfer to the operating room, i.e. how different forms of simulation training influence operative outcome in the operation room. As conclusion of the review for laparoscopic cholecystectomy, participants who received simulation-based training prior to conducting patient-based assessment generally performed better than their counterparts who did not have this training. This improvement was not universal for all the parameters measured, but the untrained group never outperformed the trained group. Trained groups generally made fewer errors, and had less instances of supervising surgeon takeover than participants who did not have the training (17).
Nine randomised trials of VR simulators and one of a video assisted simulator were identified to fill the inclusion criteria (Table 1) (18 –27). In addition, two earlier studies from the ASERNIP-S review were included to the analysis (28, 29). Apart from the basic skills (cutting, clip placement, grasping and orientation with the camera), simulation training included cholecystectomies at least by stages (deliberation of the cystic duct, clip placement and cutting of vessels and other anatomical structures, removal of the gall bladder). One recent study tested haptics feature (27). All reported materials were small. The study objects comprised surgical residents with varying postgraduate years, one study included medical students. In some studies (Table 1) the effects of simulator training were described mainly with outcome measures registered by the simulator device. Seven studies evaluated live cholecystectomies.
Randomised trials of simulator training of laparoscopic cholecystectomy
IG, intervention group; CG, control group; VR, virtual reality; MIS, minimally invasive surgery; ES, error score / tissue damage; EM, economy of movement; TH, tissue handling; NA, not assessed; LC, laparoscopic cholecystectomy; GOALS, global operative assessment of laparoscopic operative skills: depth perception, bimanual dexterity, efficiency, tissue handling, and overall competence; SA, skills assessment / overall competence; FLS, Fundamentals of Laparoscopic Surgery; IH, instrument handling; GRS, global rating score.
Identified from the ASERNIP-S report 2007
The results from all studies were rather consistent: the use of a simulator improved motor skills of the surgical trainees when assessed by various technical or quality measures (Table 1). The duration of the procedure became shorter along with the training, but this result was not considered to be of high importance. The use of haptics was problematic and did not improve the performance in one study (27). In all, those who had trained with simulators usually performed better than those who had not, at least when measured by some of the indicators used in each study. In one study no difference for the benefit of training was observed (18). Six of seven studies that evaluated live cholecystectomies resulted in significantly better overall performance and/or fewer errors in the trained group when compared to the untrained participants. In the study of Ahlberg et al. this difference persisted during the first ten laparoscopic cholecystectomies in the operation room. (22).
Four comparative studies were evaluated (30 –33). In three of them the goal was to develop a simulation training curriculum, and in one study the effects of a four day simulation course in addition to a basic skills course was evaluated. As a conclusion, in order to achieve the best benefit from simulation training, it needs to be incorporated into a training curriculum with pre-set goals of learning.
BLACK BOX TRAINING
From the separate literature search concerning black boxes and VR simulators, seven randomised studies were assessed (Table 2) (10, 34 –39). The study subjects consisted mainly of medical students — there were no surgical trainees in the groups. The training included basic laparoscopic skills rather than the complete performance of cholecystectomy. The methods and results varied so much that it was not possible to make definite conclusions. More or less, the technical skills were improved with both tested training methods.
Virtual reality (VR) simulators compared to black box trainers in learning laparoscopic skills, randomised controlled trials.
IG, intervention group; CG, control group; VRMSS, virtual reality motor skills simulator; VR, virtual reality; MIS, minimally invasive surgery; ES, error score / tissue damage; TH, tissue; handling; LC, laparoscopic cholecystectomy; ns, not significant; GS, Global score; ICSAD, the Imperial College Surgical Assessment Device comprised of a motion-tracking system and software.
METHODOLOGICAL VALIDITY OF ORIGINAL STUDIES
The quality of studies varied considerably. Nine of twelve studies concerning simulators were estimated to be of adequate quality (18, 20 –26, 29), as well as three of seven black box studies (36, 38, 39). Various studies did not have comparable simulation methods or outcome measures, which hampered drawing firm conclusions. Furthermore, all but two studies (26, 29) lacked the power calculation, and in most studies the number of participants was so small, that it might have affected the statistical analysis.
DISCUSSION
Our systematic review focussed on assessment of VR simulators with outcome measures applicable to live laparoscopic cholecystectomies. Disappointingly, only few studies provided measures transferable to clinical settings, and no long-term results could be found of the impact of simulator training on patient outcome. Thus, the evidence is scarce concerning the clinical benefits. In order to clarify the relationship between VR simulators and less expensive primitive trainers, a completion search was performed. The box trainers were typically tested among medical students, whereas the studies of VR training were undertaken among surgical residents.
Inexpensive simple black box or video trainers seem to work well in teaching the basic skills of laparoscopic surgery (26, 36, 39), and VR simulation did not unambiguously seem to bring further advantage. These box trainers are simple, durable and cheap and suit for teaching technical details in the early stages of surgical training. Typically their use requires presence of a tutor to achieve faultless performance. Full procedures may be trained in black box models, but the technical implementation is often demanding, requiring animal specimens and continuous supervision. Therefore, these more complex box models are best suited for, e.g. different training courses. Using cadavers and animal models is expensive and usually subject to license.
Computer based laparoscopic VR simulators seemed to be of benefit when applied with proficiency goals and as part of structured training program with expert tuition combined with systematic and structured evaluation (14, 40, 41). Simulator training diminishes the number of technical mistakes, thus enabling the surgeon to concentrate more on the other aspects of surgery, such as decision making and fluent performance. There is a theoretical foundation for the use of simulators in order to improve patient safety and skills learning. The supposition is, based on the experience from aviation, that both motivation and learning benefit from the simulation being as realistic as possible. More studies are needed about the usefulness of haptic features in VR simulators.
So far there are only preliminary data on the transference of skills learned with laparoscopic simulators to live operations (14, 17, 22, 40). Regarding laparoscopic cholecystectomies, there were only seven small randomised studies on how and to which extent the motor skills learned with simulators were transferred to live surgery (18, 22, 23, 25, 26, 28, 29). It is not known what is the minimum of technical properties needed in a simulator to achieve the training goals. In order to measure improvement in skills, an automatic recording and feed-back system is required. Further research to assess the characteristics of optimal simulator training and its timing in the curriculum is required.
It is difficult to evaluate the clinical benefit of simulators as many different aspects influence the learning of surgical techniques over a long training period. In spite of attempts to standardise evaluation criteria, the meaning and importance of different variables remains unclear. Research data support the assumption that any kind of simulator training improves motor skills especially in the early stages of surgical training. This could help adapt to more complicated tasks, enhance cognitive capabilities, and eventually reduce operative risks. Simulator training programs with proficiency targets improved the consistency of training and resulted in better performance (25).
According to skills learning theories, the learner is actively involved in learning process and he/she constructs new knowledge on top of his/her own prior knowledge and conceptions (42). The learner's own responsibility and reflection on learning and performance are essential in the learning process. The tasks and exercises have to be suitable to each learner's level of skill, and the role for instructor is to facilitate and support the learner. Competence from the novice level to medical (surgical) expertise is reached through several stages (8), (43). Especially in the beginning, concrete experiences and ‘learning by doing’ are fundamental (44). With simulation, the learner encounters realistic problems and exercises, which allow training in a controlled environment outside the real-life situation but not without a connection to authenticity (42, 45). Simulation allows opportunities to explore, succeed and even fail without the risk of harming patients.
Learners should be enabled to follow their own progress and receive adequate feedback and evaluation (46). A simulator can be connected to an interactive social context in which facilitation and peer support promotes learning. Training results and expert feedback should be available, especially in the beginning, in order to avoid misapprehension (47). The exercises should include partial tasks, supportive feedback and several goals to strive for (46). Expertise in complex visuo-motor skills can not be reached without repetition and deliberate practice (48, 49). Authentic operating situations are likely to cause extra stress for beginners and slow down the learning process (50). The consequences of stress diminish with experience (50). In controlled simulator training the learner may fully concentrate on learning. In addition, the new generation simulators enable the learning of decision making, the flow.
Besides training, simulators may be used for other purposes. For surgeons infrequently doing laparoscopic procedures, simulators may provide a means of keeping their skills up-to-date. Surgeon's technical skills could also be regularly assessed. Thus, a valid license could be required from all surgeons doing laparoscopic surgery. It has also been suggested, that simulators could be used for choosing persons more suitable for laparoscopic surgery. Any of these concepts are not very familiar to the surgical profession, at least in the Nordic countries. Validity for all these issues needs further investigation.
Very little is known about the cost benefit of VR simulators. The cost of a black box or video simulator is only a small proportion of that of high technology simulators. Besides the initial cost of buying a simulator and appropriate software, there are continuous expenses from its use and maintenance. E.g. simulator training during working hours with tuition causes expenses of the salaries of both trainees and trainers. However, if simulator training leads to faster and safer operations, this could result in savings as well. One severe bile duct injury may necessitate even a liver transplantation giving extra costs up to 100 000 euro (51). The value of the ethical considerations can't be evaluated.
Several questions remain unanswered. In the report of ASERNIP-S, research recommendations were given that remain valid. It recommends further research into the transfer of skills acquired via simulation-based training to the patient setting. Future studies could explore the nature and duration of training required to deliver the greatest transfer effect, the stage of training at which trainees receive maximum skill transfer benefits from different forms of simulation, the effect of different levels of mentoring during the training on transfer rates, and changes in staff productivity as a result of simulation-based training (17). All these issues need further high quality studies before we can conclude, whether the possible benefit gained by a more complex simulator justifies the higher price. Neither it is still not known, whether training results get better with increasing complexity of the simulator, although new providers and more advanced simulators turn up on the market and the use of them in surgical training is increasing.
All meaningful practise leads to the learning of new skills. The lack of research data and the use of simulators in different training settings made it difficult to evaluate the effect of simulators on surgical training. To synthesise the data, current simulators and learning programs might be of best value in settings where basic surgical training is given, i.e. during the first years of residency (in central hospitals in the Finnish training system). So far the availability of regular simulator training is not equally distributed, but several courses utilising simulators are being arranged in Finland and abroad. The council of Managed Uptake of Medical Methods (MUMM) consisting of representatives from all hospital districts has recently recommended that simulation training within appropriate education programs should be available for surgical residents. For saving investment cost, collaboration between hospital districts is encouraged (52).
Footnotes
ACKNOWLEDGEMENTS
We wish to thank Tiina Lehmussaari, National Institute for Health and Welfare, for technical help.