Abstract
Study Design:
Longitudinal survey.
Objective:
It remains important to align competence-based objectives for training as deemed important by clinical fellows to those of their fellowship supervisors and program educators. The primary aim of this study was to determine trainee views on the relative importance of specific procedural training competencies. Secondarily, we aimed to evaluate self-perceived confidence in procedural performance at the commencement and completion of fellowship.
Methods:
Questionnaires were administered to 68 clinical fellows enrolled in the AOSNA fellowship program during the 2015-2016 academic year. A Likert-type scale was used to quantify trainee perspectives on the relative importance of specific procedural competencies to their training base on an established curriculum including 53 general and 22 focused/advanced procedural competencies. We measured trainee self-perceived confidence in performing procedures at the commencement and completion of their program. Statistical analysis was performed on fellow demographic data and procedural responses.
Results:
Our initial survey response rate was 82% (56/68) and 69% (47/68) for the follow-up survey. Although most procedural competencies were regarded of high importance, we did identify several procedures of high importance yet low confidence among fellows (ie, upper cervical, thoracic discectomy surgery), which highlights an educational opportunity. Overall procedural confidence increased from an average Likert score of 4.2 (SD = 1.3) on the initial survey to 5.4 (SD = 0.8) by follow-up survey (P < .0001).
Conclusions:
Understanding trainee goals for clinical fellowship remains important. Identification of areas of low procedural confidence and high importance to training experience will better guide fellowship programs and supervisors in the strategic delivery of the educational experience.
Introduction
Surgical education continues to evolve with increasing focus on competence-based approaches to training. 1 The establishment of the Accreditation Council for Graduate Medical Education (ACGME) marked the era of external periodic review to ensure educational standards for medical training. Dr Frank Eismont addressed this issue in his Presidential Address to the Cervical Spine Research Society published in 1996, 2 summarizing the progress and called on spinal societies to further direct the content of fellowship programs to ensure adequate training for future spine surgeons. Herkowitz et al 3 in 2000 suggested further specific guidelines for spinal training that included both resident- and fellowship-level considerations. Their article highlighted the need for specific objectives and a curriculum at the fellowship level to ensure that the training offered in these programs meet the needs of trainees who eventually enter independent spinal surgical practice. In 2006, Herkowitz et al 4 reported on the American Board of Orthopedic Surgery examination results for trainees completing both ACGME accredited and nonaccredited spine fellowship programs. They showed higher achievement among graduates of accredited programs and concluded that this supported the value of periodic review of programs to ensure maintenance of high-quality education.
In 2017, most spine surgeons in training around the world continue to develop their expertise through either an orthopedic or neurosurgical residency training program. The increasing number and complexity of spinal procedures means that most trainees now seek to undertake at least 1 or 2 years of spine fellowship training. Orthopedic trainees may graduate residency with less confidence in their ability to perform certain spinal procedures than their neurosurgical counterparts, 5 and there is significant variation in case numbers across different residency programs. Neurosurgical residents in the United States complete more spine cases than orthopedic residents, but orthopedic residents are often exposed to a greater numbers of spinal deformity cases. 6 European neurosurgical graduates have been shown to have self-reported incomplete competence managing the spectrum of spinal disorders. 7 Studies across Canada have shown that procedural competency expectations during residency in spine have declined, placing a greater emphasis on quality fellowship-level training. 8 With increasing emphasis being placed on fellowship-level training in spine surgery, there remains a relative paucity of literature available on this subject. 9
A recent survey of 289 AOSpine Europe members highlighted the differences in self-reported competence between surgeons who had undertaken a year of fellowship training and those who had not. There were significant differences observed between the groups, with no significant differences observed between orthopedic and neurosurgical-trained spine surgeons. This led the authors to conclude that all spine surgeons should consider spine fellowship training and that ideally this should be guided by a formal curriculum. 10
Against this background the Canadian Spine Society (CSS) developed a syllabus for spinal fellowship training. 11 A consensus-based syllabus of cognitive and procedural competencies were established by a national panel of fellowship educators, program directors, and academic and community surgeons including both orthopedic and neurosurgical representation. A modified-Delphi methodology was used to reach agreement on these competencies. The purpose of this study was primarily to determine the perspectives of the 2015-2016 cohort of clinical AOSpine North America (AOSNA) fellows on the relative importance of each procedure to their goals of training. Secondarily, we also determined trainee self-perceived procedural confidence in performing spinal procedures included in the syllabus at the beginning as well at end of their AOSNA fellowship year.
Materials and Methods
A questionnaire was developed and ratified by a working group comprising spine surgical and education members of the AOSpine North America Executive, Fellowship Committee, Education Committee, including representatives of the CSS. The survey comprised background demographic details for each fellow and additional questions asking fellows to rank key procedural competencies from the established procedural fellowship syllabus. Fellows involved in pediatric spinal surgery fellowships were presented with additional questions relating to the specific pediatric procedural section of the syllabus. Fellows were asked at the beginning of their AOSNA fellowship to rank their confidence in performing each of the procedures using a 7-point Likert-type scale. They were also asked to rate how important they thought that competency item was to them as a component of their overall goals for fellowship training (Table 1). A repeat questionnaire ranking procedural confidence was administered toward the end of their fellowship year.
Likert Scales for Importance and Confidence.
This study consisted of 68 eligible fellows enrolled in the AOSpine North America fellowship for 2015-2016. This is a competitive fellowship with currently 26 AOSpine fellowship sites across North America, each program as selected through a peer-review process as assessed against published criteria. 12 The questionnaire (Appendix 1, available in the online version of the journal) was administered using SurveyMonkey and was sent out with a covering message from the AOSpine North America Fellowship Committee in August 2015. The follow-up questionnaire was administered in June 2016. Two weeks after the initial survey request a reminder was issued to fellows who had not yet completed the survey. A final reminder was then issued at 4 weeks. Survey questions included pediatric procedures (for those completing fellowships with exposure to pediatric spinal disorders), general spine procedures, cervical spine procedures, thoracic spine procedures, lumbosacral spine procedures, and oncology and other advanced focused spine procedures. These procedural competencies matched those described by Larouche et al. 11 More broadly, the syllabus contained a list of more general spine procedural competencies (n = 53) as well as more advanced/focused procedural competencies (n = 22).
Results were analyzed and summarized using statistical methods by a biostatistical expert. Analyses were performed using SAS 9.3 (SAS Institute, Cary NC). Based on clinical relevance, importance and confidence scores were dichotomized into high versus low importance (Likert scale 6 or 7 vs 1 through 5) and high versus low confidence (Likert scale 4 through 7 vs 1 through 3; Table 1). Questions with high importance but low confidence were tested for by determining whether low confidence was significantly different from expected by comparing 95% confidence interval of likelihood of low confidence to random chance of 50%.
Differences by specialty (Neurosurgery vs Orthopedic Surgery) were compared for importance and confidence by testing whether the difference of percentage of high importance and high confidence between Neurosurgical versus Orthopedic residency training was significantly different from zero.
The effect of length of spine training during residency on procedural confidence results was also examined by dichotomizing residency spine training time into less than or equal to 6 months and more than 6 months. We then tested confidence scores between the groups using parametric Student’s t tests. Mean confidence scores for general syllabus procedural questions versus advanced syllabus questions were compared using parametric paired Student’s t tests, since the same fellow scored both general and advanced questions.
Questions from both the general and advanced syllabi were grouped into dimensions based on region (eg, cervical spine) or disease (eg, oncology). The importance and confidence scores were then compared using the χ2 test. When 25% of the cells had expected counts of less than 5, the Fisher’s exact test was used.
Finally, overall initial confidence was compared with confidence at follow-up using the nonparametric Wilcoxon 2-sample test.
Results
Our initial survey of 68 AOSpine North America fellows for the year of 2015-2016 achieved an 82% response rate (56/68) with 70% completing all required questions. The response rate for the follow-up survey was 69% (47/68). The mean age of the fellows respondents was 32.9 years (range = 28-50). Fifty-two respondents were males and 2 were females. Forty fellows had completed residency in the United States, 4 in Canada, and 12 outside North America. Forty-three (77%) of the fellows came from orthopedic residency programs, and 13 (23%) were from neurosurgical residency programs. Forty-eight (86%) respondents were entering their first year of spine fellowship training, 6 (10%) were starting their second year of fellowship training, and 2 (4%) had already completed at least 2 years. The subspecialty interests are reported in Table 2. Fifty percent of respondents anticipated practice that would involve pediatric spine with 24 (43%) of fellows involved in fellowships focusing on both adult and pediatric spine training. No fellowships were focused solely on pediatric spine and 32 (57%) focused purely on adult spine training. Forty-seven (84%) of fellows reported a significant research focus as part of their fellowship.
Subspecialty Interests of the Clinical Fellows.
In general, good concordance was noted between syllabus items and fellow’s perceived importance with more variable replies in some focused or advanced areas (eg, spinal injections). Procedural confidence had a greater spread over the possible scale. We were able to summarize values for importance and confidence based on our 7-point Likert-type scales. The data for each question is presented in Appendix 2 (available in the online version of the journal). Most items were considered of high importance.
Several procedures were identified as being of high importance (Likert scale 6 or 7) and low confidence (1 through 3) among fellows. These items are presented in Table 3. These skills include pediatric procedures such as fusion for spondylolisthesis, surgical management of congenital anomalies, and traumatic conditions of the pediatric spine. General procedures include upper cervical instrumentation, including sublaminar wiring techniques, odontoid screw fixation, and revision decompression of the cervical spine and thoracic discectomy, were considered important with low experience.
List of Procedures With High Importance and Low Confidence.
Significant differences were observed in importance responses comparing orthopedic and neurosurgical residency-trained fellows. A higher importance rating among neurosurgical trainees for use of intraoperative navigation systems (P = .0001), primary extradural tumor management (P = .0001), primary intradural tumor management (P < .0001), and management of syringomyelia (P < .0001) was observed.
Significant differences in confidence were also identified between orthopedic and neurosurgical residency-trained fellows. These included increased confidence among neurosurgical trainees for multilevel anterior cervical corpectomies (P < .0001), revision cervical spine decompression (P < .0001), posterior and lateral thoracic spine approaches (P < .0001), primary extradural tumor management (P = .002), primary intradural tumor management (P < .0001), management of syringomyelia (P < .0001), and dorsal column stimulator placement (P = .0001).
Low importance responses (Likert scale 1-5) were compared by dimension (question groups) and significant differences were seen comparing orthopedic and neurosurgical trainees in the areas of the thoracic spine (P = .044), spine oncology (P < .001), and Questions 90 to 97 (Appendix 1, P = .048). There was also a significant difference in low importance ratings in spine oncology (P < .001) when comparing the number of months of spine training during residency. There were no other significant differences in “low importance” (Table 4).
Low Importance Dimensionsa,b.
a“Low Importance” = importance score ranges from 1 to 5.
Q5 = What residency program have you completed?
Q6 = How many months of spine training did you undertake during residency?
Q7 = In what year of spinal fellowship training are you?
bStatistical significance is based on 2-sided test with P value ≤.05, which is in boldface. When χ2 test may not be a valid test with 25% of the cells having expected counts less than 5, Fisher’s exact test is used.
Greater residency exposure to spine was associated with greater general procedural confidence. Twenty-six fellows had less than or equal to 6 months of residency experience in spine surgery, and 24 fellows had more than 6 months of residency experience in spine surgery. The overall average confidence score for all competency questions (Questions 12 to 97; Appendix 1) was 3.6 (±1.1) for fellows with up to 6 months of spine training compared with 4.8 (±1.1) for fellows with more than 6 months of spine training. The difference of overall confidence score was statistically significant with p value = .0002 (Table 5).
Low Confidence Dimensionsa,b.
a“Low Confidence” = confidence score ranges from 1 to 3.
Q5 = What residency program have you completed?
Q6 = How many months of spine training did you undertake during residency?
Q7 = In what year of spinal fellowship training are you?
bStatistical significance is based on 2-sided test with P ≤ .05, which is in boldface. When χ2 test may not be a valid test with 25% of the cells having expected counts less than 5, Fisher’s exact test is used.
Statistically significant differences in “low confidence” responses (Likert ratings 1-3) were observed between neurosurgical and orthopedic trainees in the cervical spine, thoracic spine, and spine oncology dimensions. The number of months spent in spine training during residency also produced significant differences in low confidence ratings across almost all dimensions (Table 5).
The overall self-perceived confidence score increased significantly from an average of 4.2 (SD = 1.3) in the initial survey to 5.4 (SD = 0.8) in the follow-up survey (P < .0001).
Discussion
This study was motivated by the desire to improve fellowship education for spinal surgeons and examines spine fellows’ perceptions regarding their own procedural competence and the importance of specific procedures. With the development of a fellowship-level syllabus of competency-based objectives for spinal surgeons in training by the CSS, 11 we have now determined the views of fellows on the importance of various elements. We measured their confidence at the commencement of their AOSpine North America fellowship year performing various procedures. This study identified gaps in confidence among both neurosurgical and orthopedic residency-trained surgeons as they began their fellowship year but did support an overall increase in confidence through fellowship training. These gaps may direct the educational opportunities offered by spine fellowship programs. More important, this study highlighted several key procedural skills that trainees have low confidence in performing but regard as important skills to acquire (Table 3). If these procedures are ones not commonly encountered at a fellowship training site, this list may provide a useful guide for spine fellowship program directors when planning educational opportunities that may include attending Instructional Course Lectures, cadaveric workshops, and other simulation-related courses. Reading programs have also been shown to be beneficial 13 and could potentially be guided by these key items.
This study achieved an acceptable initial and follow-up response rates. We did note that all but 2 of the respondents were male and do recognize the ongoing opportunity to better capture gender considerations specific to training objectives. Both neurosurgical and orthopedic fellows were included in the survey group with the majority (75%) of fellows having completed orthopedic residency training. We also recognize this other potential bias in related data capture and evaluation. Most (80%) of the fellows had completed residency training in North America as part of the AOSNA fellowship opportunity, although 20% were graduates of various international residency programs. This supports the international nature of fellowship training and suggests a role for future international discussions on standards, curricula, and opportunities in spine fellowship education.
The majority (85%) of fellows were starting their first year of spine fellowship training and so the self-reported confidence is likely to represent the level at graduation from residency rather than that of individuals with subspecialty training. This survey also provides us useful insight into the difference in confidence between neurosurgical and orthopedic residency graduates. More than 40% of fellows had exposure to pediatric spinal disorders through their fellowship, giving a meaningful sample group for pediatric procedural competency questions.
As anticipated, enhanced residency exposure to spine training reflected in a greater self-perceived procedural capability at the commencement of fellowship training. This is educationally interesting in that further reviewing spine case numbers for graduating residents 6 may further explain some of the differences observed in studies comparing neurosurgical and orthopedic resident confidence with spinal procedures. 5 Despite higher caseloads, a recent survey of members of the American Association of Neurological Surgeons demonstrated learning needs around adult spinal deformity even among some practicing surgeons. 14 What remains somewhat less clear in the spine field is the number of surgical cases required to be “competent, “versus that required to be “proficient,” or to be considered an “expert.” Many residency programs focus on basic competency at the end of training. Proficiency and expertise clearly also needs to be considered during fellowship and transition into independent surgical practice. This motivates ongoing research relating to this learning curve.
Limitations of this study include a relatively small sample group although a good response rate with longitudinal follow-up makes the results more meaningful. In this study, we utilized established fellowship procedural competencies from a Canadian (CSS) developed syllabus, and recognized that there is some variation in clinical practice and fellowship training between different countries regionally, nationally, and internationally. In the CSS syllabus development, materials were derived an environmental scan from a number of international sources, including fellowship-level educational materials from AOSpine International. Finally, self-perceived confidence in the performance of procedures as determined by trainees may potentially differ from independently measured procedural performance. Understanding trainee goals for clinical fellowship education remains important. Identification of areas of low procedural confidence and high importance to training experience will better guide fellowship programs and supervisors in the strategic delivery of the educational experience. Residency exposure to spine surgery appears to enhance self-perceived procedural competence at the commencement of fellowship and there does appear to be some differences comparing background residency specialty training, which needs to be considered in the ongoing learning needs of clinical fellows.
Supplemental Material
Supplemental Material, GSJ747574_suppl_mat - The Importance of Determining Trainee Perspectives on Procedural Competencies During Spine Surgery Clinical Fellowship
Supplemental Material, GSJ747574_suppl_mat for The Importance of Determining Trainee Perspectives on Procedural Competencies During Spine Surgery Clinical Fellowship by Antony H. Bateman, Jeremie Larouche, Christina L. Goldstein, Daniel M. Sciubba, Theodore J. Choma, Brandon Lawrence, Joseph Cheng, Michael G. Fehlings, Scott J. Paquette, and Albert J. M. Yee in Global Spine Journal
Footnotes
Acknowledgments
Our special thanks to the many fellows who completed the survey and to Chi Lam for her administrative assistance. The authors also acknowledge and thank Mr Wei Xiong for his contributions to biostatistical analyses and Ms Ala Bankowska for her administrative assistance.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work has been supported by the Education and Fellowship Committees of AOSpine North America and the Fellowship Committee of the Canadian Spine Society.
Supplemental Material
The supplemental material is available in the online version of the article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
