Sage Journals: Discover world-class research

Abstract

Study Design:

Prospective randomized study.

Objectives:

To define the impact of an inexpensive, user-friendly, and reproducible lumbar pedicle screw instrumentation bioskills training module and evaluation protocol.

Methods:

Participants were randomized to control (n = 9) or intervention (n = 10) groups controlling for level of experience (medical students, junior resident, or senior resident). The intervention group underwent a 20-minute bioskills training module while the control group spent the same time with self-directed study. Pre- and posttest performance was self-reported (Physician Performance Diagnostic Inventory Scale [PPDIS]). Objective outcome scores were obtained from a blinded fellowship-trained attending orthopedic spine surgeon using Objective Structured Assessment of Technical Skills (OSATS) and Objective Pedicle Instrumentation Score metrics. In addition, identification of pedicle breach and breach anatomic location was measured pre- and posttest in lumbar spine models.

Results:

The intervention group showed a 30.8% improvement in PPDIS scores, compared with 13.4% for the control group (P = .01). The intervention group demonstrated statistically significant 66% decrease in breaches (P = .001) compared with 28% decrease in the control group (P = .06). Breach identification demonstrated no change in accuracy of the control group (incorrect identification from 32.2% pre- to posttest 35%; P = .71), whereas the intervention group’s improvement was statistically significant (42% pre- to posttest 36.5%; P = .0047).

Conclusions:

We conclude that a concise lumbar pedicle screw instrumentation bioskills training session can be a useful educational tool to augment clinical education.

Keywords

surgical education bioskills training spine simulation training sawbones pedicle screw insertion simulation

Introduction

While orthopedic and neurosurgical programs aim to provide a comprehensive curriculum to physicians-in-training to ensure competence in surgical skills and decision making, external pressures including work-hour restrictions, declining budgets, and medicolegal ramifications of surgical complications can limit these experiences. The performance of complex skills and tasks, including surgical skills, demonstrate a “learning curve” and require frequent repetition to ensure accuracy and consistency of the task.¹ Simulated training sessions are commonly utilized for skill acquisition and early development of complex technical tasks. For example, Gonzalvo et al concluded that approximately 40 to 80 pedicle screws were needed before a spine fellow was able to achieve reproducible accuracy consistent with attending staff skill.¹

Bioskills training modules can be practical and effective forms of education within the time and budgetary constraints of many surgical training programs. While several studies utilizing cadaveric training modules have demonstrated improvements in pedicle screw placement for trainees,^2,3 the time and cost required can be prohibitive for residency programs. With budgetary and time constraints in mind, the purpose of our study was to define the effectiveness of an inexpensive, user-friendly, readily deployable, and reproducible lumbar pedicle screw training module and evaluation protocol that can be readily implemented into residency training programs in order to augment the clinical education of orthopedic and neurosurgical physicians-in-training. We hypothesize that the implementation of a concise training session would lead to improvement in simulated surgical skills performance compared with self-directed study.

Methods

A total of 19 participants were enrolled from an orthopedic training program at a single tertiary academic medical center. The study design met the criteria for an institutional review board exemption at our institution and participants gave informed consent. Participants were stratified into 3 groups based on level of experience (medical students, PGY 1-3, and PGY 4&5) to facilitate block randomization that would control for participant level of experience. Blinded assessors were unaware of control (n = 9) or intervention (n = 10) group assignment.

All participants performed a pretest consisting of pedicle screw placement in the right-sided pedicles of L1 through L5 of a Sawbones normal anatomy lumbar spine model in a prefabricated spine holder, simulating a standard posterior open view. Standard instrumentation and surgical tools were provided and participants were allowed to select the screw size and length. The dorsal exposure was limited to the posterior elements, and foam tape was placed over both the midline elements and transverse processes, obscuring direct visualization of the pedicle or vertebral body while allowing visualization of the pedicle screw start point (Figure 1). Participants were given 30 minutes to complete the task (Figure 2). Additionally, participants completed a pedicle screw breach detection station. A lumbar spine Sawbones model with previously prepared pedicle screw tracts with either breaches at random locations or no breach was used to test participants’ ability to properly identify the presence or absence of a breach with a ball-tip probe. The breaches were made using a 2 mm drill bit at the appropriate start point for pedicle screws but headed in an aberrant trajectory in order to simulate a Lenke probe breach. The size or depth of breaches was not standardized for created breaches. Responses were subsequently scored as “correct” or “incorrect,” with “correct” scores given for answers either correctly identifying no breach or correctly identifying the location of the breach (Figure 3).

Figure 1.

Representative lumbar Sawbones model for pedicle instrumentation bioskills module. Posterior spinal approach simulated with foam tape covering spinal canal and anterior structures.

Figure 2.

Pedicle instrumentation bioskills simulation laboratory arrangement.

Figure 3.

Lumbar Sawbones model for breach identification with prefabricated pedicle screw tracts with anatomical breaches.

Following the pretest, the control group participants were instructed to read standard texts of their choosing for 20 minutes. The intervention group underwent a 20-minute training module consisting of an interactive PowerPoint presentation created by the authors demonstrating the performance of pedicle screw placement and technique demonstration using sawbones models (see Supplemental Material for PowerPoint; available in the online version of the article). The posttest evaluation for both groups was then performed on the left sided pedicles of L1 through L5 of the same model. Following the posttest, the pedicle breach detection test and written quiz were administered to all participants. Total completion time for the study was approximately 75 minutes. The cost of the Sawbones lumbar spine model was approximately US$72 per participant.

Participants’ performances of lumbar pedicle screw placement were evaluated using a modified Objective Structured Assessment of Technical skills (OSATS) scale, which has been previously utilized to assess proficiency with surgical tools, technical skills, and performance of the procedure.^4,5 The participants completed a Physician Performance Diagnostic Inventory Scale (PPDIS) following the pre- and posttests, a survey given to assess the participants’ self-reported knowledge and skill. Third, the pedicle screw models were directly inspected for accuracy of screw placement and each level was categorized as “breach,” defined as direct visualization of the pedicle screw or identification of breach with probing after screw removal, versus “no breach.” Following data collection, the subsection scores of OSATS and PPDIS for each participant’s pre- and posttest were recorded as composite scores (eg, a score of “2” for each of the 5 OSATS subsections would be reported as a score of “10”; see Supplemental Material for OSATS and PPDIS grading forms; available in the online version of the article). In order to determine the size of effect resulting from the intervention or control group, the mean change in participants’ composite scores from pre- to posttesting for OSATS and PPDIS metrics were analyzed. A 2-sample t test was used to compare the mean change from pre- to posttest scores for OSATS, PPDIS, and written quiz measures within the control and intervention groups. The breach scores and breach identification station results for the control and intervention groups were treated as categorical variables and analyzed using a Bowker’s test modification of the χ² test.

Results

The intervention group reported a significant improvement of 30.8% in PPDIS scores (mean +5.3, SD 2.98), compared with 13.4% for the control group (mean +2.0, SD 2.60) (P = .01). While the intervention group demonstrated a 37.1% improvement of OSATS scores (mean +4.2, SD 4.05) compared with the control group’s improvement of 13.6% (mean +1.89, SD 2.47), this was not statistically significant (P = .41; Table 1).

Table 1.

Statistics for OSATS and PPDIS Outcome Metrics.

	Group	N	Mean Pretest Composite Score	Mean Posttest Composite Score	Mean Change in Score	SD	P Value
OSATS	Control	9	12.111	14	1.889	2.472
	Intervention	10	11.3	15.5	4.2	4.05	P = .4101
PPDIS	Control	9	15.222	17.222	2	2.598
	Intervention	10	17.3	22.6	5.3	2.983	P = .0104

Abbreviations: OSATS, Objective Structured Assessment of Technical Skills; PPDIS, Physician Performance Diagnostic Inventory Scale.

The intervention group demonstrated a significant reduction in total breaches from pre- to posttesting (Bowker’s test; P = .001), while the control group demonstrated no significant differences (Bowker’s test; P = .06). The intervention group demonstrated 66% decrease in breaches (22% pretest vs 7% posttest) compared with a 28% decrease in the control group (26% pretest vs 17% posttest). Additionally, groups showed a no significant difference in performance on a 6-question written quiz on the technique of pedicle screw placement (P = .07; Table 2).

Table 2.

Pedicle Screw Breaches χ² Tables.

		Pretest
Control		No Breach	Breach	Total
Posttest	No breach	14	13	27	Bowker’s test; P = .0593
	Breach	5	13	18
	Total	19	26	45

		Pretest
Intervention		No Breach	Breach	Total
Posttest	No breach	25	18	43	Bowker’s test; P = .0011
	Breach	3	4	7
	Total	28	22	50

Similarly, the intervention group displayed a significant difference with correctly identifying anatomic breach location with ball-tip probe in separate prefabricated lumbar sawbones. Incorrect responses for the breach identification station increased from pre- to posttesting for the control group (32.2% to 35%, Bowker’s test P = .71), while the intervention group showed a reduction of incorrect responses (42% to 36.5%, Bowker’s test P = .0047; Table 3). Subgroup analysis of breach site identification suggests that the improvement seen in the intervention group is from improved identification of “no breach” (pretest 86.7%, posttest 96.7%) and “medial breach” (pretest 78%, posttest 92%). Identification of anterior, inferior, and multiple breaches demonstrated the lowest accuracy for both groups, averaging 10% to 30% with minimal changes from pre- to posttesting. The majority of the incorrect answers for breach identification testing were misclassified as “no breach,” accounting for 75.4% and 65.7% of the overall incorrect answers from the control and intervention groups, respectively.

Table 3.

Breach Identification Station χ² Tables.

		Pretest
Control		Incorrect	Correct	Total
Posttest	Incorrect	48	15	63	Bowker’s test; P = .7055
	Correct	13	104	117
	Total	61	119	180

		Pretest
Intervention		Incorrect	Correct	Total
Posttest	Incorrect	72	8	80	Bowker’s test; P < .0001
	Correct	24	96	120
	Total	96	104	200

Discussion

Spine surgery requires a highly technical set of surgical skills with a known increased propensity for technical error and complications during the learning process.^{6

–15} Simulation training demonstrates value for trainees through protected development of basic skills, with the goal of providing a foundation to prepare for high-risk real-world experiences.^1,16,17 The results of our study propose that a concise, inexpensive, well-designed training module can improve participants’ technical skills and proficiency with simulated performance of lumbar pedicle screw placement. Participants enrolled in the intervention group reported significant improvements in subjective understanding and self-reported improvements in the technical skill of pedicle screw placement (PPDIS). Furthermore, the intervention group demonstrated a significant reduction in breach rate with placement of pedicle screws as well as correct identification of pedicle breaches when compared with the control groups. While greater improvements in technical skills (OSATS) were seen for the intervention group compared with the control group, this did not reach statistical significance.

Historically, spine bioskills modules have had difficulty demonstrating significant improvement for its participants.^18,19 Using 10 subjects including residents and medical students, Sundar et al demonstrated a reduction in suboptimal screw placement after a sawbones and cadaver training module.^20

–23 Similarly, Harrop et al noted no statistically significant differences in pre- and postdidactic scores and in the OSATS using a Sawbones model for a posterior laminectomy with 8 subjects.²⁴ However, recent studies of orthopedic simulation programs have been able to demonstrate significantly improved cervical lateral mass and thoracic screw placement.^7,24 Gottschalk and colleagues reported that a 3D simulation-training group of 15 residents exhibited significantly improved trajectory of cervical lateral mass screw placement.²⁵

Several limitations were encountered in the development and implementation of this study. Participants may have demonstrated “testmanship” during screw placement by selecting undersized screws to reduce breach rates. Additionally, the transferability of subjective and performance skills obtained during a Sawbones training lab to clinical performance has yet to be proven and was not specifically addressed in our study. Furthermore, we were not able to assess the durability of the intervention effect as posttesting occurred immediately after the intervention. While the OSATS metric has been extensively utilized and validated for the evaluation of resident’s performance of technical skills,^10

–15 its use as a tool to evaluate the efficacy of simulation laboratories has only been recently described.^6,8,9,26 Anderson and colleagues suggested that any study that utilized the OSATS as a stand-alone assessment measure ignored critical objective assessments regarding the end product of the tested technical procedure.²⁷ For this reason, we supplemented the OSATS evaluation with direct evaluation of pedicle screw placement. Additionally, although the PPDIS can experience response bias from the Hawthorne effect (where participants modify responses or activity due to the awareness of being monitored), the outcome metric has been used in similar medical education studies and provides insight on participants’ perceived efficacy and value of the bioskills simulation training.^28,29 Furthermore, these assessment tools seem to be best suited for parametric distributions of technical skills and may exhibit floor/ceiling effects while assessing participants with very advanced or very minimal proficiency for the technical skills assessed. Additionally, the use of novel evaluation metrics precluded our ability to perform power analysis and sample size determination. Last, although our sample size (n = 19) represented the majority of our training program, this limited subgroup statistical analyses.

Despite the aforementioned limitations, we believe Sawbones simulation training can contribute positively to surgical education. The improvement in trainees’ subjective and objective skills assessments with a brief Sawbones training session suggests acquisition of essential fundamental surgical skills can be efficiently and effectively conducted in the simulation laboratory. Furthermore, trainees reported both a high level of satisfaction with simulation training and subjectively improved procedural skills. We believe the value of Sawbones simulation for pedicle screws is to improve proficiency with normal pedicle anatomy, surgical instruments, and pedicle screw technique in a controlled and low-risk environment. Although no clinical correlation of learned surgical skills was conducted, we believe simulation training is safe and effective for teaching core surgical skills, laying the foundation for trainees to learn complex anatomy and pathology with subsequent operating room experiences.

Conclusion

A standardized, cost-effective pedicle screw bioskills training module with Sawbones simulation can be a useful surgical resident and student educational tool, leading to significantly improved scores in residents’ subjective self-assessment of performance (PPDIS), improvement in breach identification, and decrease in pedicle screw placement breach rate, while trending toward improvement in OSATS. We believe that the observed improvement in participant skills has educational relevance and that similar simulation training modules can be powerful tools to augment the trainee’s surgical education.

Supplemental Material

Supplemental Material, GSJ743505_suppl_mat - The Effectiveness of Bioskills Training for Simulated Lumbar Pedicle Screw Placement

Supplemental Material, GSJ743505_suppl_mat for The Effectiveness of Bioskills Training for Simulated Lumbar Pedicle Screw Placement by Barrett S. Boody, Sohaib Z. Hashmi, Brett D. Rosenthal, Joseph P. Maslak, Michael H. McCarthy, Alpesh A. Patel, Jason W. Savage and Wellington K. Hsu in Global Spine Journal

Supplemental Material

Supplemental Material, GSJ743505_suppl_mat_ppt - The Effectiveness of Bioskills Training for Simulated Lumbar Pedicle Screw Placement

Supplemental Material, GSJ743505_suppl_mat_ppt for The Effectiveness of Bioskills Training for Simulated Lumbar Pedicle Screw Placement by Barrett S. Boody, Sohaib Z. Hashmi, Brett D. Rosenthal, Joseph P. Maslak, Michael H. McCarthy, Alpesh A. Patel, Jason W. Savage and Wellington K. Hsu in Global Spine Journal

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Michael H. McCarthy, MD

Supplemental Material

The supplemental material is available in the online version of the article.

References

Gonzalvo

Fitt

Liew

, et al.

The learning curve of pedicle screw placement: how many screws are enough?

Spine (Phila Pa 1976). 2009;34:E761–E765. doi:10.1097/BRS.0b013e3181b2f928.

Ericsson

. Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med. 2004;79(10 suppl):S70–S81.

Ericsson

Nandagopal

Roring

. Toward a science of exceptional achievement: attaining superior performance through deliberate practice. Ann N Y Acad Sci. 2009;1172:199–217. doi:10.1196/annals.1393.001.

Tortolani

Moatz

Parks

Cunningham

Sefter

Kretzer

. Cadaver training module for teaching thoracic pedicle screw placement to residents. Orthopedics. 2013;36:e1128–e1133. doi:10.3928/01477447-20130821-13.

Bergeson

Schwend

DeLucia

Silva

Smith

Avilucea

. How accurately do novice surgeons place thoracic pedicle screws with the free hand technique? Spine (Phila Pa 1976). 2008;33:E501–E507. doi:10.1097/BRS.0b013e31817b61af.

Moktar

Popkin

Howard

Murnaghan

. Development of a cast application simulator and evaluation of objective measures of performance. J Bone Joint Surg Am. 2014;96:e76. doi:10.2106/JBJS.L.01266.

Luciano

Banerjee

Bellotte

, et al. Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery. 2011;69(1 suppl operative): ons14–ons19. doi:10.1227/NEU.0b013e31821954ed.

Kyaw Tun

Granados

Mavroveli

, et al. Simulating various levels of clinical challenge in the assessment of clinical procedure competence. Ann Emerg Med. 2012;60:112–120. doi:10.1016/j.annemergmed.2012.01.036.

Van Heest

Putnam

Agel

Shanedling

McPherson

Schmitz

. Assessment of technical skills of orthopaedic surgery residents performing open carpal tunnel release surgery. J Bone Joint Surg Am. 2009;91:2811–2817. doi:10.2106/JBJS.I.00024.

10.

Hopmans

den Hoed

van der Laan

, et al. Assessment of surgery residents’ operative skills in the operating theater using a modified Objective Structured Assessment of Technical Skills (OSATS): a prospective multicenter study. Surgery. 2014;156:1078–1088. doi:10.1016/j.surg.2014.04.052.

11.

Kundhal

Grantcharov

. Psychomotor performance measured in a virtual environment correlates with technical skills in the operating room. Surg Endosc. 2009;23:645–649. doi:10.1007/s00464-008-0043-5.

12.

Niitsu

Hirabayashi

Yoshimitsu

, et al. Using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg Today. 2013;43:271–275. doi:10.1007/s00595-012-0313-7.

13.

Hadley

Lam

Briceno

Luerssen

Jea

. Use of a formal assessment instrument for evaluation of resident operative skills in pediatric neurosurgery [published online August 28, 2015]. J Neurosurg Pediatr. doi:10.3171/2015.1.PEDS14511.

14.

Argun

Chrouser

Chauhan

, et al. Multi-institutional validation of an OSATS for the assessment of cystoscopic and ureteroscopic skills. J Urol. 2015;194:1098–1105. doi:10.1016/j.juro.2015.02.2959.

15.

Buerkle

Rueter

Hefler

Tempfer-Bentz

Tempfer

. Objective Structured Assessment of Technical Skills (OSATS) evaluation of theoretical versus hands-on training of vaginal breech delivery management: a randomized trial. Eur J Obstet Gynecol Reprod Biol. 2013;171:252–256. doi:10.1016/j.ejogrb.2013.09.015.

16.

Sclafani

Kim

. Complications associated with the initial learning curve of minimally invasive spine surgery: a systematic review. Clin Orthop Relat Res. 2014;472:1711–1717. doi:10.1007/s11999-014-3495-z.

17.

Lee

Yeo

Soeharno

Yue

. Learning curve of a complex surgical technique: minimally invasive transforaminal lumbar interbody fusion (MIS TLIF). J Spinal Disord Tech. 2014;27:E234–E240. doi:10.1097/BSD.0000000000000089.

18.

McGaghie

Issenberg

Cohen

Barsuk

Wayne

. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86:706–711. doi:10.1097/ACM.0b013e318217e119.

19.

Gallagher

Seymour

Jordan-Black

Bunting

McGlade

Satava

. Prospective, randomized assessment of transfer of training (ToT) and transfer effectiveness ratio (TER) of virtual reality simulation training for laparoscopic skill acquisition. Ann Surg. 2013;257:1025–1031. doi:10.1097/SLA.0b013e318284f658.

20.

Chitale

Ghobrial

Lobel

Harrop

. Simulated lumbar minimally invasive surgery educational model with didactic and technical components. Neurosurgery. 2013;73(suppl 1):107–110. doi:10.1227/NEU.0000000000000091.

21.

Ghobrial

Anderson

Chitale

Campbell

Lobel

Harrop

. Simulated spinal cerebrospinal fluid leak repair: an educational model with didactic and technical components. Neurosurgery. 2013;73(suppl 1):111–115. doi:10.1227/NEU.0000000000000100.

22.

Ray

Ganju

Harrop

Hoh

. Developing an anterior cervical diskectomy and fusion simulator for neurosurgical resident training. Neurosurgery. 2013;73(suppl 1):100–106. doi:10.1227/NEU.0000000000000088.

23.

Sundar

Healy

Kshettry

Mroz

Schlenk

Benzel

. A pilot study of the utility of a laboratory-based spinal fixation training program for neurosurgical residents. J Neurosurg Spine. 2016;24:850–856. doi:10.3171/2015.8.SPINE15119.

24.

Harrop

Rezai

Hoh

Ghobrial

Sharan

. Neurosurgical training with a novel cervical spine simulator: posterior foraminotomy and laminectomy. Neurosurgery. 2013;73(suppl 1):94–99. doi:10.1227/NEU.0000000000000103.

25.

Gottschalk

Yoon

Park

Rhee

Mitchell

. Surgical training using three-dimensional simulation in placement of cervical lateral mass screws: a blinded randomized control trial. Spine J. 2015;15:168–175. doi:10.1016/j.spinee.2014.08.444.

26.

Bradley

Moktar

Maxwell

Wedge

Murnaghan

Kelley

. A reliable and valid objective structured assessment of technical skill for the application of a Pavlik harness based on International Expert Consensus. J Pediatr Orthop. 2016;36:768–772. doi:10.1097/BPO.0000000000000557.

27.

Anderson

Long

Thomas

Putnam

Bechtold

Karam

. Objective Structured Assessments of Technical Skills (OSATS) does not assess the quality of the surgical result effectively. Clin Orthop Relat Res. 2016;474:874–881. doi:10.1007/s11999-015-4603-4.

28.

Berthelot

Le Goff

Maugars

. The Hawthorne effect: stronger than the placebo effect? Joint Bone Spine. 2011;78:335–336. doi:10.1016/j.jbspin.2011.06.001.

29.

Gasco

Holbrook

Patel

, et al. Neurosurgery simulation in residency training: feasibility, cost, and educational benefit. Neurosurgery. 2013;73(suppl 1):39–45. doi:10.1227/NEU.0000000000000102.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB

7.46 MB