Subjective vs Computerized Assessment of Surgeon Skill Level During Mastoidectomy

Abstract

This pilot study examines the use of surgical instrument tracking and motion analysis in objectively measuring surgical performance. Accuracy of objective measures in distinguishing between surgeons of different levels was compared to that of subjective assessments. Twenty-four intraoperative video clips of mastoidectomies performed by junior residents (n = 12), senior residents (n = 8), and faculty (n = 4) were sent to otolaryngology programs via survey, yielding 708 subjective ratings of surgical experience level. Tracking software captured the total distance traveled by the drill, suction irrigator, and patient’s head. Measurements were used to predict surgeon level of training, and accuracy was estimated via area under the curve (AUC) of receiver operating characteristic curves. Key objective metrics proved more accurate than subjective evaluations in determining both faculty vs resident level and senior vs junior resident level. The findings of this study suggest that objective analysis using computer software has the potential to improve the accuracy of surgical skill assessment.