Learning from Machine Learning: Advancing from Static Images to Dynamic Video-Based Quantification of Facial Palsy

Abstract

Background:

An automated method to accurately quantify facial function from videos has been a long-standing challenge in facial palsy (FP) management.

Objective:

To compare the accuracy of a Python open-source machine learning algorithm (Python-OS) to a standard image-based analysis tool (Emotrics) to track facial movement among patients with FP, as measured by error rates.

Methods:

Landmarks were generated on patient with FP images using Python-OS and Emotrics and on patient videos using Python-OS. Weighted error rates were calculated and compared between algorithms using analysis of variance tests.

Results:

Overall major error rates were 50.3%, 54.3%, and 9.2% for the Emotrics image, Python-OS image, and Python-OS video analyses (p < 0.001). Compared to image analyses, Python-OS video analysis had higher accuracy across all facial features (p = 0.03) and FP severities (p < 0.001). Video analysis allowed us to distinguish FP-specific temporal patterns; the linear relationship between right and left oral commissure movements in normal function (R = 0.99) became nonlinear in flaccid (R = 0.75) and synkinetic (R = 0.72) FP.

Conclusion:

We report high relative accuracy of dynamic FP quantification through Python-OS, improving the clinical utility of AI-aided FP assessment.