Sage Journals: Discover world-class research

Abstract

Background

Educational videos are a potent resource for the learning of surgical skills among different study cohorts. However, there is limited evidence on the effectiveness of different educational video interventions and their features.

Methods

A systematic search of MEDLINE (via PubMed), Embase (via OVID), Cochrane libraries and Clinicaltrials.gov was performed from inception to 28/02/21. Studies included were not limited by date of publication, studies aiming to assess the impact of video-based interventions in the direct acquisition of surgical skill were included. Eligible studies were analysed based on study type, type of video intervention, method of assessment and period of education. The educational impact of the studies was also assessed as per Messick’s framework for testing validity of evaluation methods and McGhagie’s model for analysing translational outcomes.

Results

22 studies were deemed suitable for inclusion, of which 14/22 (63.6%) demonstrated a significant improvement in knowledge/skills following the video-based teaching interventions, 3/22 (13.6%) studies demonstrated an improvement in trainee satisfaction scores. A recurrent limitation of the included studies was the lack of validation of selected assessment methods. None of the included studies scored on all 5 parameters of validity as defined by Messicks validity framework. Furthermore, none of the included trials were conducted for long enough to indicate direct changes to patient outcomes resultant from educational methods.

Conclusion

Video-based surgical education is effective in learning surgical skills within different levels of surgical training; however, superior study quality and follow-up is required to determine which aspects of video-based interventions are most impactful.

Keywords

surgical education surgical skills video based surgical education surgical video surgery

Background

To date, the current model by which surgeons are trained relies heavily on apprenticeship-based Halstedian frameworks,¹ through observation trainee surgeons acquire the knowledge, technical skills and critical judgement to successfully operate on patients. Although surgical expertise is integral to healthcare, challenges exist in the training of competent surgeons such as the implementation of work time directives,² reduced opportunities for performing core procedures^3,4 and budget cuts in healthcare organisations limiting investments into surgical training.⁵

With the advent of new video recording technologies, and improved computer systems, opportunity stands for innovation in surgical education. The use of educational videos offers many significant advantages including remote learning on demand, cost effectiveness, saved time and transport fees for healthcare professionals and students. Furthermore, with the development of validated checklists in the assessment of surgical skills, the opportunity lies for the extension of video-based learning to assessment and further.⁶ The utility of videos in learning surgical skills became increasingly apparent during the COVID-19 pandemic.⁷

The aim of this study is to compare the effectiveness of video-based teaching interventions to alternative teaching methods, such as text and live workshop demonstrations. Moreover, it aims to evaluate the manner in which implementation of video-based teaching interventions is most effective in the acquisition of surgical skills by healthcare professionals.

Methods

This review was conducted according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analyses) guidelines⁸ and registered on “PROSPERO” the international prospective register of systematic reviews prior to the study commencement (Registration ID: CRD42021233836).

Eligibility

Inclusion for research analysed in this systematic review was limited to primary research. All studies with original data relating to surgical skill acquisition through the utilisation of video technology were included. Inclusion criteria for study participants were medical students, surgical trainees, surgeons of all grades and specialities, healthcare students and healthcare professionals. The control group was the cohort receiving alternative methods of training in the acquisition of surgical skills. Papers were included unrestricted by publication date however study inclusion was limited to those which could be obtained in the English language if the mentioned criteria were met. Studies that did not utilise video in the direct demonstration and teaching of a surgical skill or procedural steps were not included. Studies regarding the improvement of surgical skill following video-based feedback, patient education or live streaming the surgical teaching were not included, as this review is assessing the use of video in the initial learning and acquisition of surgical skill in healthcare professionals/students.

Search Methods

3 Databases were searched for this systematic review: MEDLINE (Via PubMed), Embase (via OVID) and Cochrane libraries, Clinicaltrials.gov was also searched for eligible ongoing studies. The search was conducted using the terms: Video* AND surgical AND teaching OR skill OR education. The listed databases were searched from inception date to 28/02/21. Effort was undertaken in analysing reference lists from articles meeting the inclusion criteria to identify additional eligible studies.

Study Selection

Study selection was performed by two independent authors and with a third more senior reviewer to settle disagreements during study selection. Three levels of review were completed, firstly title screening and abstract screening to eliminate ineligible articles and lastly a full analysis of the manuscript, disagreements following discussion regarding eligibility were settled by the third reviewer. Articles which met the eligibility criteria also had their reference lists screened for additional eligible articles. Table 1

Table 1.

Summary of included studies.

Study reference	Study Type	Participant information	Type of Intervention	Method of Assessment	Duration of Education	Outcomes	Validity	Level of Effectiveness	MERSQI score
N.Akl et al. ¹⁵	Prospective observational study	N = 12 Obstetrics and gynaecology residents (11), Practicing Obstetrician and gynaecologist (1)	Laparoscopic suturingPre/ post intervention assessmentDry lab instructional videoSinge study cohort, all given video intervention	Time taken to perform tasks/ number of movements	One session - 6-minute instructional video, 25 minutes to complete 5 tasks	Post video education median time to perform all timed tasks improved by 20% (115.5 seconds, p 5 .009)6 movements to 4 movements	Content = N Response Processes = N Internal structure = N Relations to other variables = N Consequences = N	LOE = 2	9.5
Chien et al. ¹⁶	Randomised controlled trial	N = 40First year medical students	Laceration repair – suturing skillsPost intervention assessmentEcho 360 Lecture capture software (overhead camera)Live workshop group vs. Video based learning group	Combination of various assessment criterion, 22-point suture task checklist	2 hours of training – 20-minute video and 1 hour and 40 minutes for practice	No significant difference in suturing scores at one and three months between Video based learning groups (P =.549) and live workshop learning (P =.8979) in student laceration repair performance	Content = 2 Response Processes = 1 Internal structure = 1 Relations to other variables = N Consequences = N	LOE = N/A	9.5
Dantas et al. ¹⁷	Randomised controlled trial	N = 3030 first year dentistry students	Dental surgeryPost intervention Assessment and evaluation trials testing retentionActive learning (interactive meeting) (1) vs. Text (2) vs. Text with video (3)Video group supplemented with reading material	17 item checklists, each test scored out of 10Assessed by one of four blinded calibrated observers	15-minute video of surgical procedure – 24-hour time period to study	Active learning group significantly better outcomes immediately (P<.05), All results similar after 60 days of first practice	Content = N Response Processes = 1 Internal structure= N Relations to other variables = 0 Consequences = N	LOE = N/A	10.5
Flores et al. ¹⁸	Randomised controlled trial	N = 32Medical students (Preclinical)	Plastic surgery techniquesPre/post intervention analysis with crossoverDigital animation vs. Textbook	Specific Validated surgery competency scale - graded in a blinded manner by two craniofacial surgeonsStudent evaluation of educational quality survey	20 minutes to review educational material	Pre intervention scores did not differ significantly between the participants (P = .74), Post intervention demonstrated significantly higher performance in the animation group compared to the text group (P < .001)Significantly higher scores in animation group across all metrics	Content = N Response Processes = 2 Internal structure= N Relations to other variables= 0 Consequences = N	LOE = 2	13.0
Hamour et al. ¹⁹	Longitudinal Cohort study	N = 6Head and neck surgery residents	Thyroid lobectomyPre/post intervention analysisVideo teaching module – 15 minutes including instructional text	Assessment sheet, Observational Clinical Human Reliability Assessment (OCHRA system)Completed by the same observing head and neck surgeon in a blinded fashion	15-minute video-based teaching module after 3 weeks	Statistically significant reduction (49%) in errors post intervention (P < .05)52% less staff takeover events (P < .05)	Content = 2 Response Processes = N Internal structure = N Relations to other variables = 0 Consequences = N	LOE = 2	12.2
Lwin et al. ²⁰	Randomised controlled trial	N = 50House surgeons	Basic surgical skillsPre/Post intervention analysisInteractive video-based instruction (IVBI) – Voice over narration and on-screen annotation vs. instructor led teaching sessions	Objective Structured Assessment of Technical Skills (OSATS) – Used pre and post interventionExit survey for satisfaction scores in IVBI group.	1 hour teaching session – 8 to 1 instructor to participants in non IVBI group	Self-directed IVBI was shown to be as effective as instructor-led teaching upon pre-defined non inferiority margin (.72 points below, 90% confidence interval: 3.8 to 5.2)All participants selected for the IVBI group “some- what agree” or “strongly agree” that IVBI should be included in the House Surgeon curriculum	Content = 2 Response Processes = N Internal structure = N Relations to other variables = 1 Consequences = N	LOE = 2	12.0
Mishra et al. ²¹	Randomised controlled trial	N = 16Ophthalmology residents	Ophthalmologic procedures: Blepharoplasty and laceration repairPost intervention analysisVideo of attending surgeon performing procedure on pic eyelid, fully narratedVideo and text (Group 1) group vs Text instructions alone (Group 2)	Time to completion and aesthetic grade (1-5 given by blinded attending Physician)Participant satisfaction survey	5 days before wet laboratory	Better aesthetic grades in video group for laceration repair (P=.038), however not for blepharoplasty (P = .492)100% of residents who completed the survey said they would like videos prior to wet laboratory and Oculoplastic procedures	Content = N Response Processes = N Internal structure = N Relations to other variables= 1 Consequences = N	LOE = 1	10.4
Patel et al. ²²	Randomised controlled trial	N = 27Obstetrics and Gynaecology residents	Laceration repairPre/post intervention written test/ post intervention practical testInstructional video supplemented with textbook chapter vs. live teaching session	Written examination – practical exam scored using a procedure checklistExaminers were not blinded	4 weeks between intervention and testing	No significant difference between live teaching group and video group – no pre intervention scores for procedureSignificant improvement pre/post intervention written scores in Post graduate year 1 residents (P < .001)	Content = N Response Processes = N Internal structure= 1 Relations to other variables= 1 Consequences = N	LOE = N/A	10.0
Lohre et al. ²³	Randomised controlled trial	N = 18Senior orthopaedic surgery residents	Reverse shoulder arthroplastyPre post intervention knowledge assessment/ Post intervention surgical skills assessmentImmersive virtual reality (IVR) group vs. Video learning (VL) group	Objective Structured Assessment of Technical Skills (OSATS) tool and Global rating scale (GRS)Participant learning assessment questionnairesThree expert surgeons acted as evaluators	Not clearly defined	IVR group significantly higher mean (SD) cumulative OSATS scores than the VL group (P < .001)GRS total and individual domain scores were not significantly different between groups	Content = 2 Response Processes = 0 Internal structure = 2 Relations to other variables =1 Consequences = N	LOE = 2	14.2
Norris et al. ²⁴	Randomised controlled trial	N = 24Junior Gynaecology residents	Assessment of laparoscopic Salpingo-oophorectomy (LSO) performancePost intervention assessmentSurgical teaching video group vs. traditional residency training group	Objective Structured Assessment of Technical Skills (OSATS; 20 points) toolLSO specific tool (30 points)Secondary outcomes: Knowledge questionnaire scores and self-assessed confidence scores	1 Week prior to date of surgery	No significant differences in mean (standard deviation) OSATS scores (P = .30) or LSO-specific tool scores (P = .24)Significant difference in mean knowledge scores/ knowledge of pelvic anatomy between the video and the traditional training (P = .01, P = .04).	Content = 2 Response Processes = 0 Internal structure = 2 Relations to other variables = 0 Consequences = N	LOE = 1	12.5
Yang et al. ²⁵	Randomised controlled trial	N = 6050 surgeons and 10 nurses	Robotic surgical skills on DV-simulator, Match board 2 (MB) and Thread the Rings 1 (TR)Pre and post intervention assessment of robotic surgical skills.Controller of events on simulator and robot (CESIR) video-based learning group (Group A) vs Instructor demonstration (Group B)	dV-Trainer automatic scoring system (M scores)	Completed over a single session/ unclear	Group A scores significantly higher post intervention in both MB and TR (P < .05)	Content = 0 Response Processes = 0 Internal structure = 2 Relations to other variables = N Consequences = N	LOE = 2	14.2
Crawshaw et al. ²⁶	Randomised controlled trial	N = 54General surgery residents	laparoscopic colorectal resection –Post intervention analysis18-minute narrated instructionalVideo group vs. independent study group	Validated Global assessment scale completed by attending surgeon	2 days	Residents in the video group scored significantly higher in total score (mean: 46.8 vs 42.3; P = .002)Less verbal support needed in video group	Content = 2 Response Processes = 0 Internal structure = 2 Relations to other variables = 2 Consequences = 0	LOE = 3	13.0
Sakamoto et al. ²⁷	Randomised controlled trials	N = 86Fifth year medical students	MicrosurgeryPost intervention analysis10-minute instructional video vs. 20-minute suturing practiceVideo group (Group 1) vs. Simulation group (Group 2)	The Nagoya University Micro-Suturing Assessment System (NUMSAS) Assessed by two blinded expert neurosurgeonsTime to complete taskNeo-five-factor inventory (FFI) personality test	1 hour	No significant difference between video-learning groups and simulator – learning group (P = .11287)High extraversion scores associated with shorter time to complete suturing assessment	Content = N Response Processes = N Internal structure = N Relations to other variables= 1 Consequences= N	LOE = N/A	10.0
Saun et al. ²⁸	Randomised controlled trial	N = 30Fourth year medical students	Chest tube insertionPre and post intervention analysis instructional videoVideo group vs. instructor/lecture group	Pre/post intervention questionnaireAdapted Objective structured clinical exam (OSCE) recorded and scored by research assistants	16 minutes (Video)/17 minutes (Didactic lecture)	No significant difference in procedural OSCE scores/questionnaire Pre- intervention.Significantly higher post-intervention questionnaire scores compared to the didactic group, demonstrating a greater acquisition of knowledge.	Content = N Response Processes = 0 Internal structure= 2 Relations to other variables = N Consequences = N	LOE = 2	11.5
Shim et al. ²⁹	Randomised controlled trial	N = 45Medical students	Vesicourethral anastomosisPost intervention analysisRobotic surgical skills DV-simulatorProctored video group (Group I) vs. Video group (Group II) vs. independent training group	Automated scoring algorithm – validated	60 minutes daily (How many days unclear), exercise repeated over 80 times	Significant difference between time taken to complete task and score >70 in video groups and independent groups (P < .001), no significant difference when comparing group I and group II, which suggests that the educational video may be as beneficial as expert proctoring (P = .291)	Content = 0 Response Processes = 0 Internal structure = 2 Relations to other variables = N Consequences = N	LOE = 2	12.0
Summers et al. ³⁰	Randomised controlled trial	N = 69First year medical students	Knot tying and suturingPre and post intervention assessmentInstructor led videotape reviewDidactic Instructor demonstration vs. videotape instruction, or vs. multimedia Computer based training instruction	Written multiple choice test, designed by National board of medical examinersPerformance based skills assessment	4 hours and 30 minutes (Including: introduction, knot tying and suturing)	CBT groups demonstrated statistically significant (P < .01) enhancement of technical skills compared with the didactic group.After 1 month, a calculated performance quotient revealed statistically significant (P < .01) improvement only in the CBT group.	Content = N Response Processes = 0 Internal structure = 0 Relations to other variables = 2 Consequences = N	LOE = 2	10.5
Takiguchi et al. ³¹	Randomised controlled trial	N = 36Medical students and 1st year trainee doctors	Endoscopic knot tyingPre/post intervention assessmentRepeated viewing of video at various speedsInstructor training (IT) group (Group A) vs IT and Self training (Group B) vs IT and cyber visual training (Group C)	Virtual reality system, MIST-VRTime taken to complete	10 – 25 minutes depending on group allocation	Time required for knot tying was significantly shorter in group C than the time in group A (P = .0375) No significant difference between groups B and CThere were no significant differences in the parameters assessed using the MIST-VR	Content = N Response Processes = N Internal structure = N Relations to other variables = 0 Consequences = N	LOE = 2	12.1
Sun et al. ³²	Randomised controlled trial	N = 20Novice dental students	Oral surgery techniquePost intervention assessment10-minute instructional videoVideo instruction group vs. text instruction group	Recording of procedure assessed according to The Arch Bar Placement Assessment Scale (ABPAS) scored from 0 -19Student perceptions questionnaire	10 minutes video/text instruction, 45 minutes examination	Post instructional skill scores not significantly different (P = .46), Significant preference for video-based instruction (P < .05)	Content = 0 Response Processes = N Internal structure = 1 Relations to other variables = N Consequences = N	LOE = 1	10.0
Xeroulis et al. ³³	Randomised controlled trial	N = 60First year medical students	Suturing and knot tying skillPre/Post intervention/retention assessmentComputer based video instruction (CBVI)Control vs. Video based vs. concurrent feedback vs. Summary feedback	Global rating scale pre/post retention trials graded by 2 blinded experts (OSATS)Imperial college surgical assessment, measured number of movements and total time	1 hour of practice	Mean GRS scores between pre- test and post-test demonstrated a significant improvement in all groups (P < .001), Mean GRS scores between pre- test and retention-test were significant only for the CBVI and summary feedback groups (P = .037).	Content = 2 Response Processes = 0 Internal structure = 2 Relations to other variables=N Consequences = N	LOE = 2	11.5
Yee et al. ³⁴	Randomised controlled trial	N = 22Surgical residents	Open carpal tunnel releasePre/post study and intervention assessmentEdited 9-minute recording of open carpal tunnel release procedureIntraoperative recording with voice over description of operative steps vs. Operative text group	OSATS graded by 3 blinded board-certified plastic surgeonsPre/post intervention questionnaires to track confidence in surgical skills	40 minutes	Surgical video group committed fewer operative errors (P = .043), and were more confident in their abilities post procedure (P = .043)	Content = 2 Response Processes = 0 Internal structure = 2 Relations to other variables =2 Consequences = N	LOE=2	14.5
Yeung et al. ³⁵	Randomised controlled trial	N = 80First and second year medical students	Laparoscopic square knot on box trainerPost intervention assessmentInstructional videoText group vs. video group	Time for completion, numbers of attempts at task, numbers of people who expressed understanding of task	10 minutes to review materials15-minute time limit on box trainer	Time for completion were not statistically different in the two groups (P = .38). However, time to review the material (407 s text v. 258 s video, P = .001), number of attempts at the task (15 text v. 5 video had n > 2 attempts, P = .01), Number who expressed understanding (35% text v. 59% video, P = .047) were significantly different	Content = N Response Processes = N Internal structure = N Relations to other variables = 0 Consequences = N	LOE = 2	10.0
Yoganathan et al. ³⁶	Randomised controlled trial	N = 40Foundation year one doctors	Single handed reef knotPost intervention assessment360 Degree video technology3D Virtual reality (VR) Video group vs 2D video group	Royal college of surgeons, Basic surgical skills course marking criteria assessed by blinded examiner	15 minutes for video, 5 minutes for practiceNo time limit with completion of the knot	Scores were significantly better in the VR Group (median knot score 5.0 vs 4.0 P = .04)25% More of the VR group cohort were able to construct a completed reef knot	Content = N Response Processes = N Internal structure = 2 Relations to other variables = 0 Consequences = N	LOE = 2	13

Data Extraction

The extraction of data from the eligible studies was completed and checked by both authors, disagreements following discussion were resolved by the third reviewer. Information extracted from the studies included: study type, type of educational video, participant information, duration of education, method of assessment and outcome data analysed for methodological validity and educational effectiveness using Messicks and McGhagie’s frameworks. For data extraction, a standard template was decided on prior to analysis of studies (Table 2).

Table 2.

Template used for data extraction.

Study Type	Participant information	Type of Intervention	Method of Assessment	Duration of Education – Length of video	Outcomes
•E.g. Cohort study, Randomised/Non-Randomised controlled trial	•N = Number of participants•Participant background	•Procedure being practiced•Pre/Post intervention assessment•Type of video•The study groups	Criteria for assessment•Range of scores•Who was involved?•Method of standardisation	•Duration of education and assessment if specified in study	•Statistical significance

Data Analysis

Eligible studies were analysed according to Messicks criterion (Table 3)^9,10 testing for validity of the study assessments on five different facets: content, response processes, internal structure, relation to other variables and consequences. Each of these parameters were graded according to the Beckman et al. rating scale (Table 4). The level of validity in each domain ranged from N, where there was no discussion of the source of validity, to two whereby data strongly supported the source of validity.¹¹ Validity scoring was performed by two independent authors, a senior author was present to reconcile any disagreements in scoring. Scores were subsequently reviewed by all co-authors prior to completion of this manuscript. The level of effectiveness (LoE) of the educational models in the respective studies were quantified using an adapted version of McGhagie’s translational outcomes, to serve video-based educational models as opposed to simulation based educational models. In the present review McGhagie’s initial translational outcomes have been extended to five outcomes (Table 5).^12,13

Table 3.

Messick’s modern concept of validity framework.

Parameter	Definition	Examples
Content	Test items are relevant and representative of the intended construct	Using expert opinions to ensure all domains are accurately represented
Response Processes	Thought processes and actions of subjects and observers are made in accordance with the intended construct	Quality control of assessments, such as in standardising test administration and minimising examiner bias
Internal Structure	Test scores across tasks can be reliably reproduced	Calculating inter-item reliability and test-retest reliability
Relations to Other Variables	Test scores correlate with external, independent measures which share a theoretical relationship	Comparing scores between groups with different levels of experience in the tested skill
Consequences	The impact of using the assessment	Determining the pass-fail score and considerations for the subject on obtaining a pass or fail

Table 4.

Rating scale by Beckman et al.

Score	Definition
N	N = no discussion of source of validity evidence presented
0	0 = discussion of source of validity but no data
1	1 = data weakly supports source of validity or is limited
2	2 = data strongly supports source of validity

Table 5.

McGhagie’s Translational outcomes of simulation-based mastery learning (adapted version).

Parameter	Definition	Examples	Rating
Internal acceptability	The trainee’s satisfaction with using the video-based learning platform	Favourable responses from feedback forms or post training survey questionnaires	Level 1
Contained effects	Changes in performance of surgical skills	Development of knowledge and/or skills as measured by a form of assessment	Level 2
Down-stream effects	Behavioural changes in the clinical context	Adopting safer patient-care practices	Level 3
Target effects	Direct changes to patient outcomes	Reduced rates of surgical complications	Level 4
Collateral effects	Changes on a wider, systemic level	Cost saving; skill retention	Level 5

Risk of Bias

Study quality was assessed and quantified using the validated medical education research study quality instrument (MERSQI),¹⁴ the MERSQI tool consists of six scoring domains created to assess different aspects of medical education research. Domains include study design, sampling, type of data, validity of assessment instruments, data analysis, and study outcomes. The maximum score per study is contingent on the study purpose, hence for this systematic review scores were standardised over a common denominator of 18 enabling comparison. Due to the nature of the studies and the heterogeneity of the study designs and outcome measures, qualitative analysis of the articles was performed.

Results

Study Selection

The search produced 2201 results, following duplicate removal 1688 studies remained for screening. The 1688 studies underwent title and abstract screening, 34 remaining studies underwent a full text screening, after a thorough analysis of the 34 articles, a further 12 articles were excluded, 22 articles remained for inclusion (Figure 1).

Figure 1.

Study selection. Taken from: PRISMA flow sheet Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal. pmed1000097.

Study Characteristics

Of the included studies 20/22 were randomised controlled trials and 2/22 were prospective observational studies. The number of intervention groups also differed between studies. The total number of participants across the included studies within this systematic review was 797.

Types of Intervention

The surgical skill focus being learnt and tested varied between studies and can be broadly divided into three categories : full surgical procedures 4/22 (20%),^19,23,26,34 general surgical skills (suturing, knot tying etc.) 11/22 (50%)^{15,16,20,22,25,27,30,31,33,35,36} and simulated surgical procedures 5/22 (22.7%).^{17,18,21,23,28,29,32} Studies also varied on method of assessment, some articles assessed skills pre intervention and post intervention 12/22 (54.5%), whereas others solely examined skills post intervention 10/22 (45.5%).

Features of Video-Based Interventions

Multiple methods were used in the delivery of video-based interventions, the most common simply included 2-dimensional video with narrations describing procedural steps which 11/22 (50%) mentioned to have utilised either through audio or on-screen narration. Animation technology was implemented in 2/22 studies (9.1%),^18,24 a further 2/22 (9.1%) also used still images throughout the procedural video to augment certain instructional steps.^31,32

Level of Effectiveness

Of the 22 papers analysed according to McGhagie’s translational outcomes,¹³ 3/22 (13.6%), studies scored an LOE of 1 indicating positive trainee satisfaction relating to the intervention, typically recorded via feedback forms.14/22 (63.6%) papers scored an LOE of 2 suggesting a statistically significant improvement in surgical skill as quantified by a form of examination, either practical or knowledge based. 1/22 (4.5%) of studies scored an LOE of 3, indicating an improvement of performance in the clinical context. Crawshaw et al.²⁶ assessed the effectiveness of incorporating a comprehensive instructional video prior to the performance of a right laparoscopic colectomy, which improved overall surgical performance and operative independence within surgical residents. Lastly, 4/22 (18.2%) of the included studies scored N/A, this was allocated when no statistically significant positive outcome was reported relating to the administration of the educational intervention. None of the articles included in this study reported a LOE of 4 or 5 whereby patient outcomes were quantifiably affected or changes witnessed on a systemic level. This LOE requires an additional level of analysis, through widescale long term implementation of the educational interventions, which was not conducted in the included studies. Rather studies assessed educational interventions on a more immediate level, such as in trainee satisfaction and improvement of knowledge and skills.

The majority of studies within this systematic review assessed outcomes through validated practical examination checklists, completed by blinded expert examiners testing multiple aspects of good surgical practice such as proper motion handling, knowledge of instruments, and flow of operation.³⁷ Subjective student experience surveys, designed to assess the allocated intervention on factors such as: student satisfaction, clarity of teaching materials and whether the participant deems the educational method effective³⁸ were also incorporated in 7/22 studies. Of the 7 studies which incorporated student surveys, 6 (85.7%) demonstrated favourable responses towards the video-based teaching intervention.

Video Based Interventions vs. Conventional Methods

Conventional teaching methods were used in 9/22 (40.9%) studies within this systematic review, defined in this context as: live workshops, textbook use and instructor led teaching sessions/lectures. In 7/9 (77.8%) of the studies comparing video-based interventions to conventional methods, a LOE of 1 or 2 was recorded, favouring video-based interventions. Overlying themes which made video-based interventions a superior modality, include the ability of the participant able to control replay speed, pause and advance to focus on certain aspects.^20,21 Further benefits of the use of educational video were in the post recording edits, educational video can be enhanced through the use of narrated diagrams, physical demonstration and the ability to highlight specific aspects augmenting understanding among participants.²⁸

Video Based Learning vs. Virtual reality

Two of the included studies within this systematic review compared typical video-based education to virtual reality,^23,36 Lohre et al.²³ assessed orthopaedic residents in performing a reverse shoulder arthroplasty, results revealed that the immersive virtual reality had a significantly higher mean OSATS scores than the video group (15.9 [2.5] vs 9.4 [3.2]; difference, 6.9; 95% CI, 3.3-9.7; P < .001). Yoganathan et al.³⁶ demonstrated similar findings, although the study focussed on acquisition of basic surgical skills as opposed to a complete procedure. Scores were significantly superior in the virtual reality (VR) group (median knot score 5.0 vs 4.0 P = .04) than the video-based intervention group, with 25% more of the cohort being able to construct a completed reef knot.

MERSQI Scores

All the included studies above were graded according to the MERSQI tool which has a maximum possible score of 18, although for certain studies adjustments were made as not all assessment fields were applicable and a common denominator of 18 was implemented. The mean MERSQI score across all studies was 11.66 (64.7%) (range = 9.5 -14.5 (52.7% - 80.6%), Standard deviation = 1.55 (8.6%)).

Validity of Evaluation Methods

Of the analysed studies within this systematic review, 14/22 (63.6%) studies reported with data strongly supporting the validity of the evaluation instrument (Table 4) on at least one of Messicks parameters (Table 3). 9/22 (40.9%) reported grade 2 validity on one parameter, 3/22 (13.6%) studies reported grade 2 validity on at least 2 parameters, and only 2/22 (9.1%) studies reported a grade 2 of validity on 3 domains. Validity was quantified through an analysis of the eligible papers and where specific information relating to the features of an evaluation instrument were not included, more information was obtained regarding the evaluation tool through the relevant paper obtained from reference reviews.

Discussion

This systematic review has demonstrated that video-based teaching methods led to improved surgical skills/trainee satisfaction in 17/22 of the studies. Previous systematic reviews investigating the use of video in surgical education conducted by Green et al.³⁹ and Ahmet et al.⁴⁰ suggest a positive impact, however the eligibility criteria for both studies was limited to medical students and residents, conversely this study analysed research relating to any healthcare profession. The analysis of students from a range of different academic backgrounds, was considered more reflective of educational effectiveness of the video-based methods in surgical skill acquisition, due to the variability in the study cohort.

Effective Implementation of Educational Video

An evident theme from this study was that video-based interventions which utilised virtual reality and simulation-based education proved more effective than standard videos. The immersive nature of virtual reality may render superior visuospatial awareness of technique, studies utilising virtual reality training had outcomes significantly superior to basic teaching video.^23,36 Sakamoto et al. comparing simulation-based teaching interventions for microsurgical skills found video-based interventions to be inferior, improvement in the simulation group may have been due to the opportunity for hands on rehearsal.²⁷

Video Based Learning vs. Conventional Methods

Several features of video were markedly advantageous to conventional methods, themes identified upon analysis included: the ability to control replay speed and to navigate through videos directly to specific parts of focus,^20,21 the highlighting of different anatomical structures, the imposition of schematic overlays and the use of audio/text narrations.²⁸ Furthermore, through video editing attention can be directed towards the most important aspects of the teaching via auditory and visual annotations, enabling learners to achieve a superior understanding of taught concepts. Video editing also enables the removal of non-essential information reducing extraneous load, which by the cognitive load theory, would increase learning potential.^41,42

Advantages of Using Videos

Video-based teaching enables standardisation of educational materials, enabling not only a consistent foundation for future clinical trials but also consistently high-quality educational content for healthcare students/professionals. An example is the New England Journal of Medicine (NEJM) video series for learning surgical procedures. Videos are designed by multiple experts, they then undergo peer review prior to dissemination. Through the use of this video series, Saun et al. proved the superiority of standardised video-based teaching methods over didactic teaching for chest tube insertion, authors speculating the basis being due to the possible variability in the teaching approaches by different instructors, being overcome by standardised video.²⁸ With advances in computer technology, trainees have access to high quality digital content at the click of a finger, creating opportunity for improvement of access to surgical education. Traditional learning of surgical techniques relies heavily on workshop-based interventions and the apprenticeship model,⁴³ however research suggests significant advantages of augmenting surgical skills training with educational video through the ability to widely disseminate surgical video via the internet, study remotely and control aspects of material review.³⁹ Video is also documented to enhance retention in the learner to a greater degree, a person typically retains 10-15% of that which is read, 10-20% of what is listened to, and 20-30% of what is viewed, however when audio and video materials are used in conjunction the retention of knowledge increases to 40-50%.⁴⁴

Disadvantages of Educational Videos

Although the employment of video-based teaching methods poses significant advantages, surgical video to date may not be able to replace traditional interactive teaching methods. Interactive teaching sessions have the additional element of live feedback and the ability to ask questions which solely relying on a teaching video would not provide.²⁸ Further studies, assessing video-based interventions with additional features such as expert feedback during practicing of surgical skills and questions and answers, may enable a more comprehensive comparison of teaching methods.

Recommendations

Most studies included in this systematic review (17/22) demonstrated improved examination scores and positive trainee perceptions resulting from video-based training. However, whether video-based education had advantages which translated to improved performance in the clinical context was undocumented, except in a study by Crawshaw et al.²⁶ The extension of studies to assess surgical performance beyond a simulated environment can support the investment into video-based educational resources and curricula for trainees. Conduction of such research could be in retrospective analysis of complications and operative time prior to video-based education and after. Alternatively, patient outcomes in a surgical setting could be investigated through a randomised controlled trial where video-based education is employed.

In addressing low validity scores, the design of checklists and study protocols (with validity criterion accounted for) would have improved external validity of included studies. The advantages of the sources of validity mentioned are evident, an example of this is in comparing a study by Chien et al.¹⁶ which scored high on content validity, with Dantas et al.¹⁷ which showed no evidence of content validity. Chien et al.¹⁶ used a scoring checklist with domains based upon predefined guidelines in published literature accounting for expert opinion, whereas Dantas et al.¹⁷ did not clearly define where test items were derived.

The establishment of a study protocol checklist, which accounts for validity and investigation of similar outcomes, would increase study quality and homogeneity allowing a more rigorous evaluation of video-based surgical education. The literature reports the existence of standardised educational video libraries, for example the NEJM video series. The uniformity in composition video features, if applied across multiple study groups, would further prove the utility of video-based education.

Limitations

Numerous recurring weaknesses were identified within the studies, firstly the low study quality as assessed by the MERSQI tool with the average quality quoted at 64.7%, Secondly, the use of validated evaluation instruments was lacking, none of the 22 studies reported validity on all five sources. Thirdly, none of the studies in this systematic review achieved higher than an LOE of 3, suggesting the lack of long term follow up. It may beneficial if future clinical trials could attempt to assess translational outcomes in terms of clinical practice. Also, multiple studies failed to specify the type of video used, as well as adjunctive features which may have been causative in the witnessed change/lack of change of assessment scores, thereby preventing a holistic analysis of video features which may have contributed to outcomes.

Due to the heterogeneity in study methodology of included papers, quantitative assessment of effectiveness of video-based interventions was not feasible and qualitative analysis of studies was performed. Studies differed drastically on multiple measures such as outcomes assessed limiting comparability. This study limited inclusion to papers which were in the English language.

Conclusion

Video based interventions proved to have a positive effect in (17/22) 77.3% of the studies analysed in this systematic review. With the advent of new video technologies, and the ease of accessibility of video-based teaching, based on the findings of this review, the creation of standardised video programmes for the teaching of surgical skills is likely to enhance current teaching methods. It is evident from this study that video-based education overcomes many limitations in acquiring procedural knowledge and was favoured by trial participants. Although video may not be able to replace the interactive aspect of learning surgical skills, an analysis of the literature proved favourable outcomes in practical and knowledge examinations. Future research in the field should ensure evaluation tools are validated, ensuring reliability of any results, and aim to assess outcomes on a wider scale outside of the immediate study setting.

Footnotes

Author Contributions

Abdullatif Aydin and Samy Cheikh Youssef contributed to the conception and design of this study, Samy Cheikh Youssef and Alexander Canning carried out study selection and assessed data from studies with Abdullatif Aydin as senior reviewer. This article was written with input from Prokar Dasgupta, Abdullatif Aydin, Nawal Khan, Samy Cheikh Youssef and Alexander Canning, and critically appraised by Prokar Dasgupta, Abdullatif Aydin and Nawal Khan. All authors read and approved the final manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Data Availability

All sources of data analysed during this study are cited in the bibliography.

ORCID iDs

Samy Cheikh Youssef

Abdullatif Aydin

References

Cameron

. William Stewart Halsted: Our surgical heritage. In: Annals of Surgery, 225; 1997. doi:10.1097/00000658-199705000-00002.

McIntyre

Winfield

TeSen

Crook

. Implementation of the European Working Time Directive in an NHS trust: Impact on patient care and junior doctor welfare. Clin Med J R Coll Physicians London. 2010;10(2):134-137. doi:10.7861/clinmedicine.10-2-134.

Toll

Davis

. More Trainees and Less Operative Exposure: A Quantitative Analysis of Training Opportunities for Junior Surgical Trainees. Bull R Coll Surg Engl. 2010;92(5):170-173. doi:10.1308/147363510x12689975699630.

Parsons

Blencowe

Hollowood

Grant

. Surgical training: The impact of changes in curriculum and experience. J Surg Educ. 2011;68(1):44-51. doi:10.1016/j.jsurg.2010.08.004.

Nagendran

Kiew

Raine

Atun

Maruthappu

. Financial performance of English NHS trusts and variation in clinical outcomes: A longitudinal observational study. BMJ Open. 2019;9(1):e021854. doi:10.1136/bmjopen-2018-021854.

Niitsu

Hirabayashi

Yoshimitsu

, et al. Using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room. Surg Today. 2013;43(3):271-275. doi:10.1007/s00595-012-0313-7.

Burke

. The impact of COVID-19 on surgical training. Bull R Coll Surg Engl. 2020;102(S1):3-3. doi:10.1308/rcsbull.tb2020.2.

Liberati

Altman

Tetzlaff

, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ. 2009;339:b2700. doi:10.1136/bmj.b2700.

Messick

. Meaning and Values in Test Validation: The Science and Ethics of Assessment. Educ Res. 1989;18(2):5-11. doi:10.3102/0013189X018002005.

10.

Noureldin

Lee

McDougall

Sweet

. Competency-Based Training and Simulation: Making a “valid” Argument. J Endourol. 2018;32(2):84-93. doi:10.1089/end.2017.0650.

11.

Beckman

Cook

Mandrekar

. What is the validity evidence for assessments of clinical teaching? J Gen Intern Med. 2005;20(12):1159-1164. doi:10.1111/j.1525-1497.2005.0258.x.

12.

Mcgaghie

. Medical education research as translational science. Sci Transl Med. 2010;2:19cm8.

13.

Mcgaghie

Issenberg

Barsuk

Wayne

. A critical review of simulation-based mastery learning with translational outcomes. Med Educ. 2014;48(4):375-385. doi:10.1111/medu.12391.

14.

Reed

Beckman

Wright

. An assessment of the methodologic quality of medical education research studies published in The American Journal of Surgery. Am J Surg. 2009;198(3):442-444. doi:10.1016/j.amjsurg.2009.01.024.

15.

Akl

Giles

Long

Magrina

Kho

. The Efficacy of Viewing an Educational Video as a Method for the Acquisition of Basic Laparoscopic Suturing Skills. J Minim Invasive Gynecol. 2008;15(4):410-413. doi:10.1016/j.jmig.2008.03.012.

16.

Chien

Trott

Doty

Adkins

. Assessing the impact of video-based training on laceration repair: A comparison to the traditional workshop method. West J Emerg Med. 2015;16(6):856-858. doi:10.5811/westjem.2015.9.27369.

17.

Dantas

Shinagawa

Deboni

MCZ

. Assessment of Preclinical Learning on Oral Surgery Using Three Instructional Strategies. J Dent Educ. 2010;74(11):1230-1236. doi:10.1002/j.0022-0337.2010.74.11.tb04997.x.

18.

Flores

Demoss

Klene

Havlik

Tholpady

. Digital animation versus textbook in teaching plastic surgery techniques to novice learners. Plast Reconstr Surg. 2013;132(1):101e-109e. doi:10.1097/PRS.0b013e3182910aa9.

19.

Hamour

Mendez

Harris

Biron

Seikaly

Côté

DWJ

. ORIGINAL REPORTS A High-De fi nition Video Teaching Module for Thyroidectomy Surgery. J Surg Educ. 2017;75:481-488. doi:10.1016/j.jsurg.2017.07.019.

20.

Lwin

Naing

, et al. Self-Directed Interactive Video-Based Instruction Versus Instructor-Led Teaching for Myanmar House Surgeons: A Randomized, Noninferiority Trial. J Surg Educ. 2018;75(1):238-246. doi:10.1016/j.jsurg.2017.06.004.

21.

Mishra

Mathai

Della Rocca

Reddy

. Improving Resident Performance in Oculoplastic Surgery: A New Curriculum Using Surgical Wet Laboratory Videos. J Surg Educ. 2017;74(5):837-842. doi:10.1016/j.jsurg.2017.02.009.

22.

Patel

Lasala

Tulikangas

O’Sullivan

Steinberg

. Use of a beef tongue model and instructional video for teaching residents fourth-degree laceration repair. Int Urogynecol J. 2010;21(3):353-358. doi:10.1007/s00192-009-1042-3.

23.

Lohre

Bois

Pollock

, et al. Effectiveness of Immersive Virtual Reality on Orthopedic Surgical Skills and Knowledge Acquisition Among Senior Surgical Residents: A Randomized Clinical Trial. JAMA Netw open. 2020;3(12):e2031217. doi:10.1001/jamanetworkopen.2020.31217.

24.

Norris

Papillon-Smith

Gagnon

Jacobson

Sobel

Shore

. Effect of a Surgical Teaching Video on Resident Performance of a Laparoscopic Salpingo-oophorectomy: A Randomized Controlled Trial. J Minim Invasive Gynecol. 2020;27(7):1545-1551. doi:10.1016/j.jmig.2020.01.010.

25.

Yang

Perez

Hubert

Hossu

Perrenot

Hubert

. Effectiveness of an integrated video recording and replaying system in robotic surgical training. Ann Surg. 2017;265(3):521-526. doi:10.1097/SLA.0000000000001699.

26.

Crawshaw

Steele

Lee

, et al. Failing to Prepare Is Preparing to Fail: A Single-Blinded, Randomized Controlled Trial to Determine the Impact of a Preoperative Instructional Video on the Ability of Residents to Perform Laparoscopic Right Colectomy. Dis Colon Rectum. 2016;59(1):28-34. doi:10.1097/DCR.0000000000000503.

27.

Sakamoto

Okamoto

Shimizu

Araki

Hirakawa

Wakabayashi

. Hands-on simulation versus traditional video-learning in teaching microsurgery technique. Neurol Med Chir (Tokyo). 2017;57(5):238-245. doi:10.2176/nmc.oa.2016-0317.

28.

Saun

Odorizzi

Yeung

Johnson

Bandiera

Dev

. A Peer-Reviewed Instructional Video is as Effective as a Standard Recorded Didactic Lecture in Medical Trainees Performing Chest Tube Insertion: A Randomized Control Trial. J Surg Educ. 2017;74(3):437-442. doi:10.1016/j.jsurg.2016.10.017.

29.

Shim

Kim

Pyun

, et al. Comparison of effective teaching methods to achieve skill acquisition using a robotic virtual reality simulator Expert proctoring versus an educational video versus independent training. Med (United States). 2018;97(51):1-5. doi:10.1097/MD.0000000000013569.

30.

Summers

Rinehart

Simpson

Redlich

. Acquisition of surgical skills: A randomized trial of didactic, videotape, and computer-based training. Surgery. 1999;126(2):330-336. doi:10.1016/S0039-6060(99)70173-X.

31.

Takiguchi

Sekimoto

Yasui

, et al. Cyber visual training as a new method for the mastery of endoscopic surgery. Surg Endosc Other Interv Tech. 2005;19(9):1204-1210. doi:10.1007/s00464-004-8236-z.

32.

Sun

Elo

Mcclendon

Park

. Comparative Efficacy of Video and Text Instructional Modalities for an Oral Surgery Technique among Dental Students. Pacific Journal of Health. 2018;1(1).

33.

Xeroulis

Park

Moulton

Reznick

LeBlanc

Dubrowski

. Teaching suturing and knot-tying skills to medical students: A randomized controlled study comparing computer-based video instruction and (concurrent and summary) expert feedback. Surgery. 2007;141(4):442-449. doi:10.1016/j.surg.2006.09.012.

34.

Yee

Padovano

Rowe

Hill

EJR

Fox

Moore

, et al. The Effect of Surgical Video on Resident Performance of Carpal Tunnel Release: A Cadaveric Simulation-Based, Prospective, Randomized, Blinded Pilot Study. Plast Reconstr Surg. 2020;145(6):1455-1463. doi:10.1097/PRS.0000000000006817.

35.

Yeung

Justice

Pasic

. Comparison of Text versus Video for Teaching Laparoscopic Knot Tying in the Novice Surgeon: A Randomized, Controlled Trial. J Minim Invasive Gynecol. 2009;16(4):411-415. doi:10.1016/j.jmig.2009.02.011.

36.

Yoganathan

Finch

Parkin

Pollard

. 360° virtual reality video for the acquisition of knot tying skills: A randomised controlled trial. Int J Surg. 2018;54:24-27. doi:10.1016/j.ijsu.2018.04.002.

37.

Martin

Regehr

Reznick

, et al. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg. 1997;84(2):273-278. doi:10.1002/bjs.1800840237.

38.

Coffey

Gibbs

. The evaluation of the student evaluation of educational quality questionnaire (SEEQ) in UK higher education. Assess Eval High Educ. 2001;26(1):89-93. doi:10.1080/02602930020022318.

39.

Green

Suresh

Bittar

Ledbetter

Mithani

Allori

. The Utilization of Video Technology in Surgical Education: A Systematic Review. J Surg Res. 2019;235:171-180. doi:10.1016/j.jss.2018.09.015.

40.

Ahmet

Gamze

Rustem

Karaborklu Argut

. Is Video-Based Education an Effective Method in Surgical Education? A Systematic Review. J Surg Educ. 2018;75(5):1150-1158. doi:10.1016/j.jsurg.2018.01.014.

41.

Lau

KHV

. Computer-based teaching module design: Principles derived from learning theories. Med Educ. 2014;48(3):247-254. doi:10.1111/medu.12357.

42.

Zhang

Zhou

Briggs

Nunamaker

. Instructional video in e-learning: Assessing the impact of interactive video on learning effectiveness. Inf Manag. 2006;43(1):15-27. doi:10.1016/j.im.2005.01.004.

43.

Wenger

. Communities of Practice: Learning, Meaning, and Identity. Syst Thinker. 1998;9. doi:10.2277/0521663636.

44.

Prinz

Bolz

Findl

. Advantage of three dimensional animated teaching over traditional surgical videos for teaching ophthalmic surgery: A randomised study. Br J Ophthalmol. 2005;89(11):1495-1499. doi:10.1136/bjo.2005.075077.

Learning Surgical Skills Through Video-Based Education: A Systematic Review

Abstract

Background

Methods

Results

Conclusion

Keywords

Background

Methods

Eligibility

Search Methods

Study Selection

Data Extraction

Data Analysis

Risk of Bias

Results

Study Selection

Study Characteristics

Types of Intervention

Features of Video-Based Interventions

Level of Effectiveness

Video Based Interventions vs. Conventional Methods

Video Based Learning vs. Virtual reality

MERSQI Scores

Validity of Evaluation Methods

Discussion

Effective Implementation of Educational Video

Video Based Learning vs. Conventional Methods

Advantages of Using Videos

Disadvantages of Educational Videos

Recommendations

Limitations

Conclusion

Footnotes

Author Contributions

Declaration of Conflicting Interests

Funding

Data Availability

ORCID iDs

References