Assessing facial weakness in myasthenia gravis with facial recognition software and deep learning

Ann Clin Transl Neurol. 2023 Aug;10(8):1314-1325. doi: 10.1002/acn3.51823. Epub 2023 Jun 9.

Abstract

Objective: Myasthenia gravis (MG) is an autoimmune disease leading to fatigable muscle weakness. Extra-ocular and bulbar muscles are most commonly affected. We aimed to investigate whether facial weakness can be quantified automatically and used for diagnosis and disease monitoring.

Methods: In this cross-sectional study, we analyzed video recordings of 70 MG patients and 69 healthy controls (HC) with two different methods. Facial weakness was first quantified with facial expression recognition software. Subsequently, a deep learning (DL) computer model was trained for the classification of diagnosis and disease severity using multiple cross-validations on videos of 50 patients and 50 controls. Results were validated using unseen videos of 20 MG patients and 19 HC.

Results: Expression of anger (p = 0.026), fear (p = 0.003), and happiness (p < 0.001) was significantly decreased in MG compared to HC. Specific patterns of decreased facial movement were detectable in each emotion. Results of the DL model for diagnosis were as follows: area under the curve (AUC) of the receiver operator curve 0.75 (95% CI 0.65-0.85), sensitivity 0.76, specificity 0.76, and accuracy 76%. For disease severity: AUC 0.75 (95% CI 0.60-0.90), sensitivity 0.93, specificity 0.63, and accuracy 80%. Results of validation, diagnosis: AUC 0.82 (95% CI: 0.67-0.97), sensitivity 1.0, specificity 0.74, and accuracy 87%. For disease severity: AUC 0.88 (95% CI: 0.67-1.0), sensitivity 1.0, specificity 0.86, and accuracy 94%.

Interpretation: Patterns of facial weakness can be detected with facial recognition software. Second, this study delivers a 'proof of concept' for a DL model that can distinguish MG from HC and classifies disease severity.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Cross-Sectional Studies
  • Deep Learning*
  • Facial Paralysis*
  • Facial Recognition*
  • Humans
  • Myasthenia Gravis* / complications
  • Myasthenia Gravis* / diagnosis
  • Software

Grants and funding

This work was funded by C2D‐Horizontal Data Science for Evolving Content with project name DACCOMPLI grant 628.011.002; LUMC Neuromuscular Fund; Target2B!.