Comparison Between Scene-Independent and Scene-Dependent Eye Metrics in Assessing Psychomotor Skills

Abstract

Objective

This study aims to compare the relative sensitivity between scene-independent and scene-dependent eye metrics in assessing trainees' performance in simulated psychomotor tasks.

Background

Eye metrics have been extensively studied for skill assessment and training in psychomotor tasks, including aviation, driving, and surgery. These metrics can be categorized as scene-independent or scene-dependent, based on whether predefined areas of interest are considered. There is a paucity of direct comparisons between these metric types, particularly in their ability to assess performance during early training.

Method

Thirteen medical students practiced the peg transfer task in the Fundamentals of Laparoscopic Surgery. Scene-independent and scene-dependent eye metrics, completion time, and tool motion metrics were derived from eye-tracking data and task videos. K-means clustering of nine eye metrics identified three groups of practice trials with similar gaze behaviors, corresponding to three performance levels verified by completion time and tool motion metrics. A random forest model using eye metrics estimated classification accuracy and determined the feature importance of the eye metrics.

Results

Scene-dependent eye metrics demonstrated a clearer linear trend with performance levels than scene-independent metrics. The random forest model achieved 88.59% accuracy, identifying the top four predictors of performance as scene-dependent metrics, whereas the two least effective predictors were scene-independent metrics.

Conclusion

Scene-dependent eye metrics are overall more sensitive than scene-independent ones for assessing trainee performance in simulated psychomotor tasks.

Application

The study’s findings are significant for advancing eye metrics in psychomotor skill assessment and training, enhancing operator competency, and promoting safe operations.