Developing Effective Automated Feedback in Temporal Bone Surgery Simulation

Abstract

Objective

We aim to test the effectiveness, accuracy, and usefulness of an automated feedback system in facilitating skill acquisition in virtual reality surgery.

Study Design

We evaluate the performance of the feedback system through a randomized controlled trial of 24 students allocated to feedback and nonfeedback groups.

Setting

The feedback system was based on the Melbourne University temporal bone surgery simulator. The study was conducted at the simulation laboratory of the Royal Victorian Eye and Ear Hospital, Melbourne.

Subjects and Methods

The study participants were medical students from the University of Melbourne, who were asked to perform virtual cortical mastoidectomy on the simulator. The extent to which the drilling behavior of the feedback and nonfeedback groups differed was used to evaluate the effectiveness of the system. Its accuracy was determined through a postexperiment observational assessment of recordings made during the experiment by an expert surgeon. Its usability was evaluated using students’ self-reports of their impressions of the system.

Results

A Friedman’s test showed that there was a significant improvement in the drilling performance of the feedback group, χ²(1) = 14.450, P < .001. The postexperiment assessment demonstrated that the system provided timely feedback (when trainee behavior was detected) 88.6% of the time and appropriate feedback (accurate advice) 84.2% of the time. Participants’ opinions about the usefulness of the system were highly positive.

Conclusion

The automated feedback system was observed to be effective in improving surgical technique, and the provided feedback was found to be accurate and useful.