A Split-Face Computer Vision/Machine Learning Assessment of Facial Paralysis Using Facial Action Units

Abstract

Background:

Accurate assessment of facial paralysis is crucial for patient management and research, yet current clinician-graded scales are limited by subjectivity and variability.

Objective:

To evaluate whether an automated video analysis pipeline using hemifacial mirroring and Facial Action Unit (AU) quantification could accurately classify facial paralysis severity (None, Incomplete, and Complete) when validated against expert clinical grading.

Methods:

A custom Python pipeline processed 112 clinical videos (100 paralysis patients, 12 controls), generating mirrored hemifacial constructs. OpenFace 2.0 extracted AU intensities from these constructs, linked to voice-command-defined action epochs. Peak expression features trained XGBoost machine learning models to predict paralysis severity (None, Incomplete, and Complete) per facial zone (upper, mid, and lower), validated against multi-expert clinical assessment.

Results:

On held-out test sets, models achieved: upper face accuracy 0.83, weighted F1-score 0.83; mid-face accuracy 0.93, weighted F1-score 0.92; and lower face accuracy 0.84, weighted F1-score 0.82. Mirrored AU intensities significantly differed across expert-defined severity groups, validating feature relevance.

Conclusion:

The automated pipeline using hemifacial mirroring and mirrored AU analysis accurately predicted facial paralysis severity from standard clinical videos.