Abstract
Eye-tracking plays a crucial role in understanding visual information processing and provides insights into cognitive behaviors. In medicine, gaze patterns help evaluate surgical expertise and training, revealing differences in cognitive workload and visual attention between experts and novices. However, these studies often rely on small sample sizes, limiting generalizability. This necessitates qualitative and quantitative methods to interpret the collective findings in the literature. While the literature contains selective or systematic reviews offering qualitative interpretations, meta-analyses is lacking. This study presents a meta analysis of eye-gaze metrics for assessing surgical expertise, focusing on fixation, and saccade related measures. Fixation frequency and duration, saccade length, and dwell time showed significant differences between expertise, but other metrics did not. Differences in task characteristics are attributed to the different results across measures. Variations in task characteristics explain these differences. advancing understanding of visual attention in medical professionals and informing training and assessment methodologies.
Eye-tracking plays a crucial role in understanding dynamics of visual information processing and provides clues on understanding cognitive behaviors including attention, working memory, decision-making, and cognitive control (Eckstein et al., 2017). In medicine, eye-tracking have been useful for evaluating proficiency level and training in surgical skills given its nonintrusive nature (Diaz-Piedra et al., 2017; Hermens et al., 2013). Multiple eye-tracking medical studies have revealed the differences in workload, visual attention between experts and novices in cognitive aspects, including information perception and processing (Berges et al., 2023; Richstone et al., 2010; Tien et al., 2015). Given the diverse applications including assessment and improvement of medical performance and expertise (Tahri Sqalli et al., 2023), exploring eye-gaze patterns over medical expertise is invaluable.
Most eye-tracking studies in medicine relied on small sample sizes, and thus findings of individual studies are hard to generalize. This necessitates qualitative and quantitative review studies to interpret the collection of latest findings. The current literature only contains selective or systematic literature reviews that provide qualitative interpretation of existing research findings (Ashraf et al., 2018; Hermens et al., 2013; Wu & Wolfe, 2019), lacking meta-analysis that would provide a quantitative perspective. Therefore, the objective of this study is to conduct a meta-analysis on eye-gaze metrics for assessing surgical expertise.
Based on their relevance to the medicine and psychology, four databases—EMBASE, PubMed, PsycNET, and TRIP—were queried for relevant scientific articles in July 2023 using a query to include three terms using AND statement: (a) eye-gaze, (b) expert and novice, and (c) medical field. This search resulted in articles published between 2008 and 2023 only. After initial search, articles were selected according to three criteria: relevance to the medical field, inclusion of eye-tracking data, and explicit comparison between two or more expertise groups with eye-gaze metrics. Articles were first filtered by the title and abstract, followed by full text examination for those three criteria. Articles are included if the contents contain comparisons of eye-gaze metrics across expertise levels and focus on the medical field. The filtered articles turned out to only pertain to diagnosis by medical imaging or assessment in surgical skills. Because of the wide variation of in how visual scenes are semantically analyzed (Blascheck et al., 2017), this meta-analysis only includes scene-independent eye-gaze metrics from each study. These metrics does not account for information in the visual scene (Deng et al., 2023; Kulkarni et al., 2023). These metrics were mainly fixation and saccade related parameters, including time to first fixation, number of fixations, fixation duration, fixation frequency, total fixation time, saccade duration, saccade frequency, saccade length, total saccade count, and dwell time. Also, task completion time and number of errors were also gathered as key performance metrics.
Initially 267 distinct articles were identified, but only 13 articles were included for the meta-analysis after screening processes. Table 1 presents the list of articles included for the final analysis.
List of Articles Included in the Meta-Analysis.
Standardized mean differences between experts and novices were calculated for all the metrics using Cohen’s d, and a random effects model was computed to estimate heterogeneity or the “true” expertise effects on the eye-gaze metrics across the findings in the articles (Riley et al., 2011). Table 2 presents the results and positive effect size means that experts show greater value than novices.
Summary of the Mean Effect Sizes for Each Metric.
Results showed that, compared to novices, experts showed significantly longer fixation duration, higher fixation frequency, shorter saccade length, and shorter dwell time. Of the two key performance metrics, experts showed significantly shorter task completion time but no significant difference in errors.
One explanation on the lack of significance for a majority of eye-gaze metrics is the lack of sensitivity to expertise. However, this explanation must be treated with caution due to the small number of articles available in the literature for metanalysis. Further, scene-dependent metrics, which were investigated in even fewer articles, were omitted. Scene-dependent eye-gaze metrics has shown better performance in differentiating expertise in one study (Deng et al., 2023; Kulkarni et al., 2023). Follow up meta-analysis will be essential to include more eye-tracking results in the future and scene-dependent eye-gaze metrics.
Another possible explanation is that studies in this meta-analysis had drastically different medical related tasks, including video watching, surgical skill training, and image diagnosis. Although meta-analysis did not used task as a factor due to lack of studies for each task, accounting for the systematic variance due to task difference may help reveal difference between expert and novice in their eye-gaze behavior. For example, Fox and Faulkner-Jones (2017) found task as a significant factor in differentiating gaze metrics between experts and novices in their meta-analysis. Many theories on expertise, including information reduction hypothesis (Haider & Frensch, 1999) and the holistic model of image perception (Kundel et al., 2007) postulated significant impact of task characteristics.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
