The Development and Execution of An Orthopaedic Sports Medicine Fellowship Surgical Skills Assessment

Abstract

Background:

Assessment of orthopaedic surgery trainees is traditionally based on subjective evaluation by faculty. The recent push for value-based health care has placed a premium on improving patient outcomes. As a result, surgical training evaluations for orthopaedic trainees are evolving to include more objective measures to evaluate competency.

Purpose:

To develop and subsequently demonstrate the efficacy of a novel surgical skills assessment for orthopaedic sports medicine fellows.

Study Design:

Case series; Level of evidence, 4.

Methods:

A team of 14 fellowship-trained orthopaedic sports medicine surgeons developed objective scoring rubrics for anterior cruciate ligament reconstruction (ACLR) and rotator cuff repair (RCR) using a modified Delphi process. Rubrics were designed based on 10 surgical steps with a grading scale (1-5) based on core competencies with a maximum summative score of 50. Fourteen orthopaedic fellows across a regionally diverse group of sports medicine–accredited fellowship programs were invited to complete both an ACLR and RCR in a surgical skills laboratory at the beginning and end of their fellowship year. Individual surgical steps, overall performance, and total procedure time were evaluated by a single sports medicine surgeon for both sessions.

Results:

Thirteen of 14 fellows completed both pre- and post-fellowship assessments. For the ACLR procedure, the pre-fellowship mean summative score was 25.4 (SD, 4.4) and the post-fellowship mean summative score was 38.6 (SD, 4.1), which was a statistically significant improvement (P < .001). For the RCR procedure, the pre-fellowship mean summative score was 26.6 (SD, 5.4) and the post-fellowship mean summative score was 38.8 (SD, 4.3), which was also a statistically significant improvement (P < .001). The mean time to completion for the ACLR procedure was 82.3 minutes (SD, 4.3 minutes) pre-fellowship, which improved to 69.7 minutes (SD, 11.6 minutes) post-fellowship (P = .002). The mean time to completion for the RCR procedure was 85.5 minutes (SD, 5.0 minutes) pre-fellowship, which improved to 76.4 minutes (SD, 7.0 minutes) post-fellowship (P < .001).

Conclusion:

This surgical skills program represents the first standardized and reproducible instrument for the evaluation of 2 arthroscopic sports medicine procedures in the United States. Orthopaedic sports medicine fellows improved significantly in aggregate over their fellowship year with regard to the ACLR and RCR. The described program has the potential to serve as both a training tool and formal orthopaedic sports medicine fellow assessment.

Keywords

surgical skills medical education arthroscopy sports medicine

Get full access to this article

View all access options for this article.

References

Angelo

St Pierre

Tauro

Gallagher

Shoulder

PBP Instructional Faculty

. A proficiency-based progression simulation training curriculum to acquire the skills needed in performing arthroscopic Bankart and rotator cuff repairs—implementation and impact. Arthroscopy. 2021;37(4):1099-1106.e1095.

de Montbrun

Roberts

Lowry

, et al. A novel approach to assessing technical competence of colorectal surgery residents: the development and evaluation of the Colorectal Objective Structured Assessment of Technical Skill (COSATS). Ann Surg. 2013;258(6):1001-1006.

de Villiers

Kent

. The Delphi technique in health sciences education research. Med Teach. 2005;27(7):639-643.

Duffield

. The Delphi technique. Aust J Adv Nurs. 1988;6(2):41-45.

Dwyer

Chahal

Murnaghan

, et al. Development of a certification examination for orthopedic sports medicine fellows. Can J Surg. 2020;63(2):E110-E117.

Ghaderi

Auvergne

Park

Farrell

. Quantitative and qualitative analysis of performance during advanced laparoscopic fellowship: a curriculum based on structured assessment and feedback. Am J Surg. 2015;209(1):71-78.

Hogle

Liu

Ogden

Fowler

. Evaluation of surgical fellows’ laparoscopic performance using Global Operative Assessment of Laparoscopic Skills (GOALS). Surg Endosc. 2014;28(4):1284-1290.

Howell

Absalom

. Consensus guidelines, Delphi methods, and evidence around anaesthetic technique for endoscopic retrograde cholangiopancreatography. Br J Anaesth. 2023;131(4):634-636.

Jacob

Duffield

Jacob

. A protocol for the development of a critical thinking assessment tool for nurses using a Delphi technique. J Adv Nurs. 2017;73(8):1982-1988.

10.

Koehler

Amsdell

Arendt

, et al. The Arthroscopic Surgical Skill Evaluation Tool (ASSET). Am J Sports Med. 2013;41(6):1229-1237.

11.

Phillips

Cheung

JJH

Whelan

, et al. Validation of a dry model for assessing the performance of arthroscopic hip labral repair. Am J Sports Med. 2017;45(9):2125-2130.

12.

Shang

. Use of Delphi in health sciences research: a narrative review. Medicine (Baltimore). 2023;102(7):e32829.

13.

Testa

Albright

Kutschke

, et al. Decreased case volume for orthopaedic sports medicine fellows during the early stages of the coronavirus disease 2019 pandemic. Arthrosc Sports Med Rehabil. 2022;4(4):e1347-e1352.

14.

Van Heest

Putnam

Agel

, et al. Assessment of technical skills of orthopaedic surgery residents performing open carpal tunnel release surgery. J Bone Joint Surg Am. 2009;91(12):2811-2817.