Motion analysis of suturing technique with Leap Motion Controller TM . Proof-of-concept

Abstract

Purpose

Leap Motion Controller is a device that can capture hand gestures and reproduce these as data comprising several parametric elements. We analyzed surgical suture motion using this device and investigated the optical methodology for clinical applications.

Methods

We recruited medical students and residents (novice group) and vascular surgeons (specialist group). The operators applied sutures once on a prosthetic graft, and the captured motion was analyzed.

Results

Ten novices, who each received procedural instruction for at least 2 h, and 10 specialists were recruited. The hand gesture consisted of four elements (roll, pitch, yaw, and wrist angle). Since “roll” was the only element in this simple suture movement that showed some difference between the two groups, we analyzed three parameters: (1) the suturing time, (2) the difference in the degree between two piercing points, and (3) slope of the roll. We found that the specialist group demonstrated significantly shorter suturing times and a larger degree of the slope.

Conclusion

Leap Motion Controller analysis with the roll revealed that the novices could use the roll motion after only 2 h of instruction; however, the suturing speed and smoothness were secondary to those of the specialists.

Introduction

Young surgeons have traditionally learned techniques by watching skilled surgeons perform operations in real-time or on video. Although their performance might have been exceptional and educational enough to pass on to posterity, there has never been an objective method to represent it. Although we can find “technical tips” in textbooks or surgical reference books, it is difficult to grasp the subtle feel of handling devices or detailed movements during the procedures.

Leap Motion Controller^TM (LMC) (Ultraleap, Inc, Silicon Valley, USA) is a portable, low-cost motion capture device designed to track joint positions, including the wrist and fingers, with an accuracy of 1/100 mm three-dimensionally.^1–3 There are some previous studies applying LMC for surgery,^4,5 however; LMC was used for the evaluation of virtual-reality simulation in the studies. We assumed that LMC would be useful for objectively explaining “master craftsmanship”, and decided to apply this device to surgical education. To the best of our knowledge, there are currently no other studies using LMC for the evaluation of a surgeon's hand motion during anastomosis. Our purpose in this study is to explore the possibility of using this device as an educational tool that numericizes and visualizes the hand motion and compared the data of various experienced surgeons and medical students.

Methods

We recruited medical students, residents, and surgeons and divided them into “novice” and “specialist” groups to participate in the monitoring of suturing a prosthetic graft. The captured data was analyzed and compared between the two groups.

Participants

The participants included medical students from the University of Tokyo and residents from the University of Tokyo Hospital whose clinical experience was 2 years or less, who comprised the novice group, as well as vascular surgeons with more than 6 years of clinical experience (average: 8 years), who comprised the specialist group. For the novice group, we instructed the suture procedure and simple off-the-job training of our department, which was described in detail previously,^6,7 for more than 2 h. The study protocol was approved by the Institutional Research Ethics Committee of the University of Tokyo Hospital (No. 11567).

Setup of prosthetic graft and Leap Motion Controller

We placed a prosthetic graft fixed on a board, and marked 2 dots 1 cm apart on the graft as a guide for the suture. The board level was 3 cm higher than the LMC level, and the graft was set from the LMC 20 cm apart (Figure 1). Each participant was oriented with graft suturing. They were instructed to pierce the graft with a needle from the right dot to the left dot, as illustrated in Figure 1. The sequence of actions was performed with the right hand only, without the use of tweezers. All the participants were right-handed. The motion of their right hand grasping the needle holder was captured with the LMC and reproduced.

Figure 1.

Setting of the Leap Motion Controller and the suturing of the prosthetic graft.

Statistics

The data were analyzed using the analysis tool on R (version.4.1.2). Mann-Whitney U test was used to compare the two groups and the differences were assessed using a two-sided test with an alpha level of 0.05. Continuous values were expressed as mean ± standard error.

Results

Ten novices who had received procedural instruction for at least 2 h, as well as 10 specialists, were recruited.

Elements of hand movements

We settled the LMC on the table with the green power light in the front and used the built-in coordinate system. The coordinate system of the LMC is for right-handed individuals, with the X-axis for the right side of the LMC, the Y-axis for the upside, and the Z-axis for the front side. Hand movements were analyzed in four elements: roll (the motion along the longitudinal axis, i.e. pronation/supination movements), pitch (the motion in the vertical plane, i.e. up-down instrument movements), yaw (the motion in the horizontal plane, i.e. side-to-side instrument movements), and wrist angle (the angle between the wrist and forearm) (Figure 2). The “roll” was programmed for rotation around the Z-axis, “pitch” for rotation around the X-axis, and “yaw” for rotation around the Y-axis.

Figure 2.

Definition of elements: Yaw, pitch, roll and wrist angle.

The captured data was reproduced as a graph of time and the degree of the angle of the elements (Figure 3). The wrist angle seemed to represent the surgeons’ style of handling surgical devices. The data on this element varied widely and no pattern could be identified. In addition, pitch and yaw did not show any significant differences among the operators, and only the “roll” element showed a significant difference in this simple suture movement between the 2 groups.

Figure 3.

Change of degrees of 4 elements in one suture.

Parameters of the “roll”

We analyzed three parameters: (1) the suturing time, (2) the difference in the degree between the two piercing points, and (3) the slope of the roll. We set one suture motion from the time point of the maximum value of the roll (t = t₀) to the point of the minimum value (t = t₁). The suturing time was calculated as t₁–t₀. Next, the difference of the degree (θ) between two piercing points (t₀ and t₁) was calculated as |θ₁ – θ₀| to represent “rotation of the wrist” Then, the slope of the role was calculated as |θ₁ – θ₀| / |t₁–t₀| to represent “smoothness of rotation” (Figure 4).

Figure 4.

Parameters of the “roll,” (a) suturing time, (b) difference of the degree between two piercing points, and (c) slope of the roll.

Differences between amateurs and professionals

There were differences in the suturing time and roll slope (Table 1). We found the specialist group demonstrated significantly shorter suture time than the novice group (2.89 ± 0.24 vs. 4.87 ± 0.35 s, respectively; Mann-Whitney U = 95; p < 0.001) and a larger degree of the slope (41.24 ± 4.86 vs. 22.38 ± 1.80 degree/sec; Mann-Whitney U = 9; p < 0.001).

Table 1.

Difference of the “roll” parameters between the novice group and the specialist group.

	Novice (N = 10)	Specialist (N = 10)	Mann-Whitney U	P-value
(1) Suturing time	4.87 ± 0.35	2.89 ± 0.24	95	<0.001
(2) Difference of the degree between two piercing points	106.29 ± 10.69	110.57 ± 6.99	42	0.5787
(3) Slope of the roll	22.38 ± 1.80	41.24 ± 4.86	9	<0.001

Discussion

Since the “watch and learn” educational concept is not socially accepted in all occupations responsible for human life, including surgery, various kinds of Off-the-job training (Off-JT) systems have been introduced in surgical education recently. The operative performance rating system (OPRS) is frequently used to evaluate the surgical procedures of young surgeons. ⁷ The OPRS is reportedly feasible, reliable, and valid,^8–10 and there are various kinds of OPRS applicable for each surgical procedure that was shown on the homepage of the American Board of Surgery. ¹¹ However, indices such as suturing graft, instrument handling, respect for tissue, time and motion, and operation flow were assumed to be subjective. ⁶ The data derived from LMC should be helpful for trainers to evaluate the procedure objectively by comparing the movements with that of a skilled specialist In the future, we may be able to feel and follow the techniques of skilled surgeons using augmented reality technology. For that purpose, we should validate the educational and clinical feedback system with objective evidence,^6,7 which is now regretfully poor. We assume that the numeric and visualized data in our exploratory study using LMC could add objective evidence.

LMC is a motion capture device with a more simple setting compared to existing marker-based devices. Ganguly et al. demonstrated the comparison of finger kinematic data between the LMC and a gold-standard marker-based motion capture system, Qualisys Track Manager (QTM). ² Both systems were time-synchronized, and the participants performed abduction/adduction of the thumb and flexion/extension movements of all fingers. The LMC and QTM were compared in both static measurements of finger segment lengths and dynamic flexion movements of all fingers. Only the correlation between the proximal interphalangeal joint of the middle and ring finger during flexion/extension was considered acceptable, and they concluded that the LMC is not suitable to replace gold-standard motion capture systems in clinical settings. In the surgical anastomosis procedure, we assumed that it is not necessary to analyze the detailed movements of the fingers but the rotation of the entire hand, which was why we used LMC in this study.

A previous systemic review discussed the state of the art commercial off-the-shelf devices in the detection of manual gestures in surgery and assessed their use as a simulation tool for motor skills teaching in minimally invasive surgery. The authors concluded that the field was still in the exploratory phase.

Among the reviewed studies, there were some studies relating to LMC. ¹² However, the most common intervention was image manipulation in surgical and interventional radiology environments, followed by interaction with virtual reality environments. We assumed that the application of LMC is not appropriate as an intraoperative supportive device during open surgery in the operation room but for basic training, as in our study.

In this study, only “roll” was focused on, which is an element that can be easily understood by cardiovascular surgeons. Since the sutured hole should be as small as possible under hemorrhagic conditions by heparinization, the rotation of the hand is believed to be critical for piercing the needle into the graft at a right angle, not diagonally. Our results only showed the smoothness of the roll in the specialist group. However, the novice group included residents who might have had some experience with suturing, and the specialist group included relatively young vascular surgeons. If the cohorts were selected to produce more of a difference in the results (i.e. medical students vs. surgeons with more than 10 years of experience) and with the more difficult Off-JT system, other parameters might be different, and the complex surgical movements can be expressed by multiple parameters.

Since the Japanese Board of Cardiovascular Surgery recommended 30 h of Off-JT for new cardiovascular specialist applicants in 2017, introducing a simple and inexpensive Off-JT system has been an urgent issue for all institutes with cardiovascular residents in Japan. The LMC only costs approximately 90 USD, and the setting with a computer is simple. The use of the LMC could help any institute to strengthen its Off-JT system. To use this device effectively, we think it is our mission to analyze the data in various settings and make simple and acceptable instructions. To improve the usability of LMC, we are now planning the next study that will involve recruiting more skilled and experienced surgeons, setting more difficult suturing conditions, and analyzing the relationship between the degree of the elements (especially “roll”) and the size of the pierced hole on the graft.

The setting of measurements was challenging and a critical point in this study. As the range of detection by LMC is within a radius of 50 cm and a central angle of 110°, we initially faced the problem that data cannot be measured when the participant's hand goes out of the imaging range. We, therefore, had to optimize the device settings, including the length of the needle holder and the distances between devices and participants.

Another limitation is the validation of the evaluating system, including the recruitment. In this study, we recruited “specialists” as vascular surgeons with more than 6 years of clinical experience. However, “specialists” would ideally be selected with factors including the number of operations, as well as years of experience as a vascular surgeon. The small number of subjects recruited was one of the limitations of this study. Although the Mann-Whitney U test revealed significant differences, the inclusion of a larger number of cases is warranted in future research to evaluate the validity of the procedures more precisely.

LMC analysis of the “roll” element revealed that the novice group could use the roll motion after only 2 h of instruction; however, the suturing speed and smoothness were secondary to those of the specialist group. The LMC may be a useful tool for describing surgical hand motion objectively and scientifically.