Learning Curves of Syllabus Otological Procedures: Monitoring Trainee Progress and Understanding Competence

Abstract

Objectives:

Otolaryngology trainees in the United King-dom undergo regular procedure-based assessments (PBAs) throughout training, however, these are rarely used to monitor progress or measure competence. The assessment tool is procedure specific and produced by the Joint Committee of Surgical Training. It consists of 6 domains: consent, planning, preparation, exposure/closure, operative technique, and postoperative care. The aim of the study was to validate PBA in assessing otolaryngology trainees and to identify the level and pace at which trainees show competence in syllabus otological procedures.

Methods:

This is a longitudinal study from October 2008 to October 2013. We analyzed all PBAs submitted by North-Thames London otolaryngology trainees, including junior or core trainees (CT) and senior specialty trainees (ST). We calculated and used the overall score (oS) mean and standard deviations to draw procedure-specific learning curves for common otological operations.

Results:

A total of 3306 PBAs from 46 trainees were analyzed, 621 were otological. PBA was highly reliable showing internal consistency (Cronbach’s Alpha: 0.921) and discriminated between different grades (CT and ST, Mann-Whitney-U: P < .001) and levels within each grade (CT1-2 and ST3-8, Kruskal Wallis: P < .001). All domains contributed to the calculated score and the most predictive was operative technique. Three procedures achieved sufficient power (n > 100, 90% confidence) to draw learning curves. The procedures were: ventilation tubes, tympanoplasty, and mastoidectomy.

Conclusions:

PBA is valid for assessing ENT trainees. Procedure-specific competency curves can be drawn to set milestones and deliver targeted training. They can help understand the pace and level at which trainees master individual otological procedures replacing arbitrary numbers currently in use.