Abstract
Background:
Simulation provides low-risk opportunities for surgical trainees to learn and practice fundamental skills. One simulation tool for orthopaedics is the Arthroscopic Knot (ArK) Trainer, which has been validated as an effective simulation tool across multiple methodologies. Previous studies have investigated the ArK Trainer in its basic form using clear plexiglass, which allows direct visualization of tissue anchors.
Purpose:
Using a mixed-methods approach, we assessed and compared junior and senior trainees’ Seoul Medical Center (SMC) knot–tying performance under direct and indirect visualization.
Study Design:
Cross-sectional study.
Methods:
Fourteen orthopaedic surgery postgraduate trainees at a single medical school were recruited to participate. Trainees tied SMC knots using the Ark Trainer under direct and indirect visualization. A mixed-methods approach was used to evaluate knot-tying proficiency and characterize participants’ approach to knot-tying. Knot-tying proficiency was evaluated using validated tools: a task-specific checklist (TSC), a global rating scale (GRS), and a proficiency scale (PS). Participants’ approach to knot-tying was characterized using Likert-type questionnaires and semistructured interviews. An α level of .10 was set a priori owing to the small pool of trainees.
Results:
The 14 participants included 7 junior residents (postgraduate years [PGYs] 1 and 2) and 7 senior residents (PGY ≥3), of whom 3 were fellows (PGY 6). Senior trainees outperformed junior trainees on both versions of the ArK Trainer: clear (GRS, P = .055; PS, P = .075) and covered (TSC, P = .05). Overall, participants performed better under direct visualization conditions (GRS, P = .05). In semistructured interviews, significantly more senior trainees discussed relying on haptic cues while tying knots under direct visualization (P = .021). The majority of trainees agreed that both versions of the ArK Trainer were realistic and appropriate practice formats for their level of training.
Conclusion:
Senior trainees were significantly more experienced than were junior trainees in arthroscopic skill and outperformed them on both configurations: direct (PS and GRS) and indirect (TSC) visualization. Experienced trainees were significantly more likely to report using tactile cues to aid knot-tying under indirect visualization. It is likely that inexperienced trainees rely more heavily on direct visualization and that the use of tactile cues may be an indicator of knot-tying proficiency. Trainees recommended progression from direct to indirect visualization configurations for inexperienced learners.
Simulation has become an essential tool in medical education. 4,6,19,20 This is particularly true because the volume of cases has declined in residency and educators are looking for more consistent learning opportunities. 18,23 While simulation is unlikely to ever completely replace the traditional apprenticeship model, 1,20 arthroscopic simulators have permitted residents to have more opportunity for hands-on skill development and progression 6,11 in a structured, low-risk, learner-centered environment. 2,20,22,24
Medical education literature has focused on defining the theoretical basis for simulated learning. There is general agreement that learning includes progression from anatomy to observation and cadaveric and simulator practice before guided learning in the operating room. 6,11,14,20 However, the optimal timing at which fundamental skills of arthroscopy, such as hand-eye coordination, triangulation, and indirect arthroscopic visualization, should be taught in the learning curve is still not fully understood. 11
To date, few studies have focused on the acquisition of skills for knot-tying under indirect visualization. 6 Arthroscopic knot-tying requires the surgeon to not only operate the camera for optimal views but also interpret 2-dimensional data on the screen to manipulate tissues and sutures using instrumentation. While the literature has postulated that indirect visualization would provide additional challenges to a learner as compared with direct visualization, no studies have shown this to date. 3,6,16
Arthroscopic knot-tying simulators have been described extensively in the literature. 4,13 The Arthroscopic Knot (ArK) Trainer, developed by the senior author (I.W.), has been validated as an effective simulation tool across multiple methodologies. 23,24 However, the ArK Trainer has been tested in only its most basic configuration. In the previously tested configuration, the ArK Trainer allowed participants to see directly through the clear plexiglass to visually guide their knot-tying. A covered (opaque) configuration is possible, which requires users to indirectly visualize their knots using a camera system, similar to true arthroscopic setups.
The purpose of this study was to apply a mixed-methods approach to assess and compare the Seoul Medical Center (SMC) knot–typing performance of junior and senior trainees under direct visualization using the clear ArK Trainer and indirect visualization using the covered ArK Trainer. Our objectives were as follows:
To compare SMC knot–tying performance between senior and junior trainees using validated scoring techniques
To understand trainees’ perceptions of each ArK Trainer configuration as a testing tool
Our hypothesis was that senior trainees would outperform junior trainees in knot-tying on both configurations of the ArK Trainer.
Methods
This study was designed as a mixed-methods prospective cohort study including orthopaedic surgery residents and fellows (postgraduate years [PGYs] 1-6) at the Queen Elizabeth II Hospital. This study was approved by a regional ethics committee.
Participant Selection
Participants were surgical trainees in orthopaedics who would ordinarily train on a surgical simulator to practice arthroscopy skills. This sample was composed of junior residents (PGYs 1-2), senior residents (PGYs 3-5), and fellows (PGY 6). A total of 14 participants were recruited, and there were no exclusion criteria. After consent was obtained, each participant completed an intake form, providing year of postgraduate training, arthroscopic exposure, and exposure to the ArK Trainer.
ArK Trainer
The ArK Trainer is a validated surgical knot-tying simulator. 24 The original, clear ArK Trainer has internal posts for knot-tying enclosed in a transparent plexiglass cover with portals for cannula and instrument insertion. The clear ArK Trainer allows users to directly visualize the knot and instruments during knot-tying. To tie a knot using the ArK Trainer, trainees must compress 2 spring-loaded hooks using 40 N of force to simulate tissue tensioning and approximation.
The modified (covered) ArK Trainer features an opaque plexiglass cover, which prevents the user from directly visualizing the knot and instruments (Figure 1). Indirect visualization was achieved using tablets (iPads; Apple Inc) with front-facing cameras and overhead cameras attached to tall stands (Figure 2). Footage from these cameras was used in evaluating participants’ performance. Both video recordings featured the knots as well as the participants’ hand movement during knot-tying.

The basic configuration of the Arthroscopic Knot Trainer. The original models had clear plexiglass, which allowed direct visualization of the suture anchors.

View of the experimental setup for several participants. Visible are the Arthroscopic Knot Trainer, an iPad (Apple Inc) with front-facing camera for indirect visualization (resting on a wooden and metal stand), and instruments. Not visible is the overhead camera, which is positioned at the top of the stand (left).
Research Plan
Study participants completed a 30-minute simulation session. Participants initially viewed an instructional video describing the instruments, knot-tying technique, and purpose of the session. The video included step-by-step instructions on how to tie an SMC knot backed up using 3 half hitches.
Participants attempted to tie SMC knots first using the clear ArK Trainer and then using the covered version. Indirect visualization was available during both knot-tying attempts. Each attempt was limited to 600 seconds. Video was recorded from 2 sources for performance evaluation: from the iPads for arthroscopy-simulated views and the overhead cameras for extracorporeal views. Participants were provided appropriate tools, such as graspers, suture cutters, cannula, and Ethicon Perma-Hand Silk No. 2-0 braided sutures.
After the completion of the simulation session, participants were interviewed. Interviews were semistructured and conducted by 2 investigators (K.M. and C.R.) familiar with the technique. Participants were prompted to reflect on the value of the ArK Trainer as a testing tool, answering questions modified from a previously validated survey. 24 They were asked 5 structured questions that were scored using a 5-point Likert-type scale. Participants were also asked to discuss their experiences with the ArK Trainer in direct and indirect visualization configurations. They answered 6 open-ended questions, with interviewers asking follow-up or clarification questions as necessary (see Appendix Table A1).
Quantitative Data Analysis
Video recordings from each participant were reviewed by 3 orthopaedic sports medicine fellows experienced in arthroscopy and SMC knot-tying and blinded to the participant experience (A.A., B.R., and E.G.). Reviewers scored the performance of each participant using 3 validated quantitative measures: the global rating scale (GRS), task-specific checklist (TSC), and proficiency scale (PS) (Table 1, Appendix Tables A2 -A4). 24 The mean scores for seniors, juniors, and all participants were calculated for the clear and covered ArK Trainer trials. Descriptive statistics and the 1-tailed independent-samples t test were used to analyze the mean difference in GRS, TSC, and PS scores. Interrater reliability was calculated using the Fleiss κ owing to the categorical nature of the rating scales. Kappa values were interpreted as follows: <0, poor agreement; 0.0-0.20, slight agreement; 0.21-0.40, fair agreement; 0.41-0.60, moderate agreement; 0.61-0.80, substantial agreement; 0.81-1.0, almost perfect agreement. Intrarater reliability was not calculated. Data were analyzed using SPSS Version 24 (IBM Corp). An α level of .10 was set a priori owing to the small pool of participants.
Measures for Evaluation of Performance
Qualitative Data Analysis
Semistructured interviews were audio recorded and transcribed verbatim. Data were analyzed using conventional content analysis methodology. 1 Data were coded into “meaning units,” which were organized into groups and subgroups based on theme. The number of data points in each meaning unit was quantified to determine trends among participants. Responses and themes were pooled across skill levels to draw conclusions about participants’ impressions of the appropriateness of direct and indirect visualization configurations of the ArK Trainer for their skill level. The chi-square test was performed to assess for significant trends in responses. Coding and analysis of data were done using NVivo (QSR International).
Results
Participant Characteristics
Fourteen surgical trainees were recruited and participated in the study. The participants were divided into 2 groups based on level of training. The junior trainees (n = 7) were junior residents (PGYs 1 and 2), who had some arthroscopic exposure but had not completed dedicated sports or arthroscopy rotations. The senior trainees (n = 7) included senior residents (PGYs 3-5) and 3 fellows (PGY 6), who had completed 1 or more rotations with significant arthroscopic training. Senior trainees reported significantly more experience with arthroscopy (P = .031), SMC knot-tying (P = .031), and the ArK Trainer (P = .031) (Table 2).
Participants by Level of Training and Reported Experience a
a ArK, Arthroscopic Knot; SMC, Seoul Medical Center.
b Statistically significant difference at P < .10.
Quantitative Results
On attempts to tie knots under direct visualization, senior trainees outperformed junior trainees when evaluated using the PS (P = .075) and GRS (P = .055). Senior trainees also outperformed junior trainees on the TSC, although this did not reach the level of significance. Under indirect visualization, the senior trainees outperformed junior trainees, although this reached the level of significance on only the TSC (P = .05). In terms of overall performance, the mean GRS score was significantly higher for junior and senior trainees using the ArK Trainer under direct visualization (P = .05). No significant difference in PS and TSC scores were found between direct and indirect visualization trials (Table 3).
Participant Performance Under Direct and Indirect Visualization, Stratified by Level of Training
a Statistically significant difference (P < .10, 1-tailed t test).
Interrater Reliability
Using the Fleiss κ, interrater reliability across the 3 rating scales was established (Table 4). The PS was calculated using rater agreement of passing or failing scores by evaluators, and there was almost perfect agreement between raters for direct and indirect visualization trials (κ = 0.93 [direct] and 0.99 [indirect]). The GRS demonstrated substantial or almost perfect agreement (κ = 0.85 [direct] and 0.90 [indirect]) and was calculated using evaluator agreement of passing or failing scores. The TSC had the lowest κ values of the 3 rating scales, but it also represented the largest number of ratings made by evaluators. Evaluators rated each participant’s trial on 20 composite tasks, totaling nearly 500 ratings. However, it still demonstrated a substantial degree of agreement.
Interrater Reliability of the 3 Rating Scales a
a Interpretation of κ values 8 : <0, poor agreement; 0.0-0.20, slight; 0.21-0.40, fair; 0.41-0.60, moderate; 0.61-0.80, substantial; 0.81-1.0, almost perfect.
Qualitative Results
SMC Knot–Tying Strategies
Participants were asked to reflect on the strategies that they used when tying knots using the ArK Trainer under direct and indirect visualization in a semistructured interview. Our aim was to identify the SMC knot–tying strategies used by junior and senior trainees.
With regard to the clear ArK Trainer, the majority of junior and senior trainees discussed the ease of direct visualization in knot-tying (Table 5). As participant 002 explained, “you can see your knot and make sure you’re seating your knot in the appropriate position.”
Participants’ Approach to Direct and Indirect Visualization With Corresponding Meaning Units a
a Chi-square analysis was performed to assess significance. ArK, Arthroscopic Knot.
b Statistically significant difference (P < .10).
When using the covered ArK Trainer, significantly more senior trainees than junior trainees described using tactile or haptic cues (P = .021) (Table 5). For example, participant 005 described a period when the knot is in the covered cannula and “you have to go by feel and have a good grasp of the knot…You have to depend on your knot and your tactile feel as opposed to actually watching the knot slide down through the clear ArK Trainer, which you don’t get to do in a real person.”
Realism and Appropriateness of ArK Trainer
Data from the semistructured interviews and Likert-type questionnaires were used to understand learners’ perceptions of the realism and appropriateness of each ArK Trainer configuration.
The majority of participants agreed that both versions of the ArK Trainer were realistic. Based on results from the Likert-type questionnaire, 57% of the trainees agreed that the clear version (direct visualization) was a realistic practice format, and 86% agreed that the covered version (indirect visualization) was realistic. Ninety-three percent of trainees agreed that the covered version was more realistic than was the clear version (Figure 3). The data from the Likert-type questions largely agreed with findings of the semistructured interviews, with most trainees (n = 11) finding the configuration with indirect visualization had more real-world fidelity. As participant 008 explained, the covered version is “more realistic of what you would use in the [operating room]. It’s more realistic to look at the screen while you are tying the knot.” The majority of participants agreed that the clear version of the ArK Trainer was easier to use than was the covered version.

Participant responses to structured questions on a Likert scale, stratified by level of training. No significant differences were detected between junior and senior trainees.
The majority of participants agreed that both versions of the ArK Trainer were appropriate practice formats for their levels of training. Based on the Likert-type questionnaire, 79% of the trainees agreed that the clear ArK Trainer was an appropriate practice format for their levels of experience, and 79% agreed that the covered ArK Trainer was an appropriate tool for them. No significant differences were detected between junior and senior trainees’ responses (Figure 3). Data from the semistructured interviews supported these findings (Table 6). All 14 participants endorsed the clear version for use by inexperienced learners, and 12 participants endorsed the covered version for use by experienced trainees. Participants’ definitions of “experienced” varied but often included those who had completed a sports rotation where residents are trained in arthroscopy. Many of the participants suggested using the clear ArK Trainer as a stepping stone for junior trainees as they progress toward the covered trainer. As participant 002 discussed, “it’s better to start with the clear version till you know your steps then you can progress to the covered version.”
Participants’ Perceptions of the Realism and Appropriateness of the Clear and Covered Versions of the ArK Trainer, With Corresponding Meaning Units a
a ArK, Arthroscopic Knot.
Participants suggested technical adaptations to the ArK Trainer to improve its utility (Table 6). Suggestions included increasing camera magnification, stabilizing the cannula in the port, stabilizing the ArK Trainer on the table, and providing more time for participants to familiarize themselves with the equipment. Senior trainees were more likely than were junior trainees to comment on the poor view from iPad and camera magnification.
Discussion
Performance
This study applied a mixed-methods approach to assess and compare SMC knot–tying performance between senior and junior residents. According to GRS and PS scores, senior trainees outperformed junior trainees under direct visualization conditions. TSC scores indicated that senior trainees also outperformed junior trainees under indirect visualization conditions. Previous experience with arthroscopy, SMC knot-tying, and using the ArK Trainer may have contributed to senior trainees’ higher scores. Existing research supports these data, as Lopez and colleagues 10 found that senior residents performed at a higher level than did junior residents and novice medical students while using a low-cost arthroscopic knot-tying simulator. Research has further demonstrated that knot-tying performance improves with practice. 14 Chong and colleagues 3 found that PGY 1 residents improved the tensile strength of their knots over a 10-week training period.
Perceptions
Data from the semistructured interviews provided insight into trainees’ perceptions of the clear and covered versions of the ArK Trainer. Participants discussed knot-tying strategies, skill acquisition, and the use of the ArK Trainer as a learning tool.
The majority of participants agreed that the clear version of the ArK Trainer was easier to use than was the covered version. Participants endorsed the possibility of progression from direct to indirect visualization using the clear and covered versions of the ArK Trainer, respectively. While studies have examined opaque box top simulators, none have compared performances across direct and indirect visualization scenarios. 15 Researchers have also attempted to test whether there are ways to ease the learning curve using visual-spatial training. Despite a variety of approaches and techniques, only limited benefits have been seen. 7,12,21 When learning to translate indirect visualization in 2 dimensions into 3-dimensional space, the learner must likely progress through predictable stages: the cognitive stage, the associative stage, and the autonomous stage. 18,20 The time necessary to progress through these stages may be variable but nevertheless cannot be entirely short-circuited.
When compared with junior trainees, significantly more senior trainees described using tactile or haptic cues when tying knots using the covered ArK Trainer. It therefore seems that experienced trainees are better able to appreciate and respond to subtle haptic feedback during the task. While more evidence is needed in this area, it suggests that integration of haptic cues is indicative a more advanced skill. This is consistent within the literature, with researchers reporting that proficiency in kinesthetic skills (eg, knot-tying) requires learning of visual and haptic cues. Ernst and Banks 5 demonstrated that humans integrate visual and haptic cues optimally to facilitate kinesthetic tasks. In a study of kinesthetic learning in surgeons, Pinzon and colleagues 16 found that muscle memory—knowledge gained through haptic learning—helps surgeons learn skills where visual feedback is limited. 17 Haptic learning may therefore aid in the progression from direct to indirect visualization during arthroscopic knot-tying.
Participants endorsed the clear ArK Trainer as a realistic simulation format and described the covered version as the most realistic version. Participants likened using the covered ArK Trainer to arthroscopy, where indirect visualization is necessary. The ArK Trainer is an accessible low-cost simulator that can be modified to suit the needs of learners as they progress.
A few participants believed that the low-fidelity aspects of the ArK Trainer limited testing performance under indirect visualization. While specific concerns can be addressed via low-cost modifications to the ArK Trainer, low-fidelity knot-tying simulators are accessible and reliable tools for learners. In a recent article, Ling et al 9 showed no performance difference between their low-fidelity arthroscopic camera and a commercial version. Furthermore, de Montbrun and MacRae 4 argued that there was no safety or performance advantage in the operating room when comparing trainees who learned using high-end or virtual reality simulators versus low-fidelity knot-tying trainers. Likewise, the majority of participants agreed that both versions of the ArK Trainer were appropriate for their learning. This setup with the ArK Trainer and tablet is an accessible low-fidelity model for learners to practice arthroscopic skills, including indirect visualization.
Limitations
This study was limited in the total enrollment of participants. Because of the low numbers of participants and the pilot nature of the study, the α level was set at .10 a priori. The results also had a large degree of in-group variability. This may have been in part because of the very limited experience of some junior trainees, some of whom actually improved on the second trial.
While the ArK Trainer was useful to discriminate among trainee skill levels, this study was not designed to show whether it would influence skill progression as a training tool. As a consequence, the conclusions that can be drawn about the role of the ArK Trainer as a training tool are limited. This is an area for future work.
Conclusion
Senior trainees were significantly more experienced than were junior trainees in arthroscopic skill and outperformed them on both configurations: direct (PS and GRS) and indirect (TSC) visualization. Experienced participants were significantly more likely to report using tactile cues to aid knot-tying under indirect visualization. It is likely that inexperienced trainees rely more heavily on direct visualization and that the use of tactile cues may be an indicator of knot-tying proficiency. Trainees recommended progression from direct to indirect visualization configurations for inexperienced learners.
Footnotes
Acknowledgment
The authors thank Sara Sparavalo and Ryland Murphy for assistance in study protocol preparation, obtaining research ethics board approval, and data collection and Jie Ma for assistance in formatting and submitting the manuscript.
Final revision submitted January 6, 2021; accepted February 18, 2021.
The authors have declared that there are no conflicts of interest in the authorship and publication of this contribution. AOSSM checks author disclosures against the Open Payments Database (OPD). AOSSM has not conducted an independent investigation on the OPD and disclaims any liability or responsibility relating thereto.
Ethical approval for this study was obtained from Nova Scotia Health Authority Research Ethics Board (ROMEO No. 1021351).
APPENDIX
Proficiency Scale a
| Proficiency formula = timing score – penalty score |
| Timing score: 600 s – time used (s) to tie 1 SMC and 3 half hitches |
| Penalty score: sum of penalties × 10 |
| 1. Approximation: 1-point penalty for each millimeter of separation >2 mm |
|
Separation of 3 mm results in an error of 3 points; a separation of ≤2 mm results in no error |
| 2. Slippage: Introduce a small pair of suture scissors into the knot in an attempt to pry it open |
|
Slippage of the knot tails is a 10-point penalty |
| 3. Disruption: Complete disruption of the knot (ie, loss of approximation of the rings) results in a 60-point penalty so that a disrupted knot receives a final score of zero |
a SMC, Seoul Medical Center.
