Sage Journals: Discover world-class research

Abstract

The Global Operative Assessment of Laparoscopic Skill (GOALS) has been shown to meet high standards for direct observation. The purpose of this study was to investigate the reliability and validity of GOALS when applied to blinded, videotaped performances. Five novice surgeons and 5 experienced surgeons were each evaluated by 2 observers during a laparoscopic cholecystectomy. Subsequently, 4 laparoscopists (V1 to V4) evaluated the videotaped procedures using GOALS. Two of the raters (V1 and V3) had prior experience using GOALS. The interrater reliabilities between video raters (VRs) and between VRs and direct raters (DRs) were calculated using the intraclass correlation coefficient. Construct validity was assessed using 2-way analysis of variance. Interrater reliability between the 4 VRs and the 2 DRs was 0.72. The intraclass correlation coefficient for the 4 VRs was 0.68 and for each VR compared with the mean DR was 0.86, 0.39, 0.94, and 0.76, respectively. All raters, except V2, differentiated between novice and experienced groups (P values ranged from .01 to .05). These data suggest that GOALS can be used to assess laparoscopic skill based on videotaped performances but that rater training may play an important role in ensuring the reliability and validity of the instrument. Experience with the tool in the operating room may improve the reliability of video rating and could be of value in training evaluators.

Keywords

intraoperative assessment laparoscopic skills surgical training video assessment feedback GOALS

Get full access to this article

View all access options for this article.

References

Sidhu RS , Grober ED , Musselman LJ , Reznick RK

Assessing competency in surgery: where to begin?

Surgery. 2004;135:6-20.

Hamilton EC , Scott DJ , Kapoor A. , et al. Improving operative performance using a laparoscopic hernia simulator. Am J Surg. 2001;182:725-728.

Adrales GL , Park AE , Chu UB , et al. A valid method of laparoscopic simulation training and competence assessment. J Surg Res. 2003;114:156-162.

Rosser JC, Jr. , Rosser LE , Savalgi RS Objective evaluation of a laparoscopic surgical skill program for residents and senior surgeons. Arch Surg. 1998;133:657-661.

Vassiliou MC , Feldman LS , Andrew CG , et al. A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg. 2005;190:107-113.

Vogt VY , Givens VM , Keathley CA , Lipscomb GH , Summitt RL Jr.Is a resident's score on a videotaped objective structured assessment of technical skills affected by revealing the resident's identity? Am J Obstet Gynecol. 2003;189:688-691.

American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for Educational and Psychological Testing. Washington, DC: American Psychological Association; 1985.

Beckmann CR , Lipscomb GH , Ling FW , Beckmann CA , Johnson H. , Barton L. Computer-assisted video evaluation of surgical skills. Obstet Gynecol. 1995;85:1039-1041.

Dath D. , Regehr G. , Birch D. , et al. Toward reliable operative assessment: the reliability and feasibility of videotaped assessment of laparoscopic technical skills. Surg Endosc. 2004;18:1800-1804.

10.

Scott DJ , Rege RV , Bergen PC , et al. Measuring operative performance after laparoscopic skills training: edited videotape versus direct observation. J Laparoendosc Adv Surg Tech A. 2000;10:183-190.

11.

Bransford J. , Donovan S. , National Research Council, Committee on Developments in the Science of Learning, National Research Council, Committee on Learning Research and Educational Practice. How People Learn: Brain, Mind, Experience, and School. Expanded ed. Washington, DC: National Academy Press; 2000.

12.

Ward M. , MacRae H. , Schlachta C. , et al. Resident self-assessment of operative performance. Am J Surg. 2003; 185:521-524.

13.

Backstein D. , Agnidis Z. , Regehr G. , Reznick R. The effectiveness of video feedback in the acquisition of orthopedic technical skills. Am J Surg. 2004;187:427-432.

Evaluating Intraoperative Laparoscopic Skill: Direct Observation Versus Blinded Videotaped Performances

Abstract

Keywords

Get full access to this article

References