End-to-end deep learning classification of vocal pathology using stacked vowels

George S Liu; Jordan M Hodges; Jingzhi Yu; C Kwang Sung; Elizabeth Erickson-DiRenzo; Philip C Doyle

doi:10.1002/lio2.1144

End-to-end deep learning classification of vocal pathology using stacked vowels

Laryngoscope Investig Otolaryngol. 2023 Aug 31;8(5):1312-1318. doi: 10.1002/lio2.1144. eCollection 2023 Oct.

Authors

George S Liu^{1

2}, Jordan M Hodges³, Jingzhi Yu⁴, C Kwang Sung^{1

2}, Elizabeth Erickson-DiRenzo^{1

2}, Philip C Doyle^{1

2}

Affiliations

¹ Department of Otolaryngology Head and Neck Surgery Stanford University School of Medicine, Stanford University Stanford California USA.
² Division of Laryngology Stanford University School of Medicine, Stanford University Stanford California USA.
³ Computer Science Department School of Engineering, Stanford University Stanford California USA.
⁴ Biomedical Informatics, Department of Biomedical Data Science Stanford University School of Medicine Stanford California USA.

Abstract

Objectives: Advances in artificial intelligence (AI) technology have increased the feasibility of classifying voice disorders using voice recordings as a screening tool. This work develops upon previous models that take in single vowel recordings by analyzing multiple vowel recordings simultaneously to enhance prediction of vocal pathology.

Methods: Voice samples from the Saarbruecken Voice Database, including three sustained vowels (/a/, /i/, /u/) from 687 healthy human participants and 334 dysphonic patients, were used to train 1-dimensional convolutional neural network models for multiclass classification of healthy, hyperfunctional dysphonia, and laryngitis voice recordings. Three models were trained: (1) a baseline model that analyzed individual vowels in isolation, (2) a stacked vowel model that analyzed three vowels (/a/, /i/, /u/) in the neutral pitch simultaneously, and (3) a stacked pitch model that analyzed the /a/ vowel in three pitches (low, neutral, and high) simultaneously.

Results: For multiclass classification of healthy, hyperfunctional dysphonia, and laryngitis voice recordings, the stacked vowel model demonstrated higher performance compared with the baseline and stacked pitch models (F1 score 0.81 vs. 0.77 and 0.78, respectively). Specifically, the stacked vowel model achieved higher performance for class-specific classification of hyperfunctional dysphonia voice samples compared with the baseline and stacked pitch models (F1 score 0.56 vs. 0.49 and 0.50, respectively).

Conclusions: This study demonstrates the feasibility and potential of analyzing multiple sustained vowel recordings simultaneously to improve AI-driven screening and classification of vocal pathology. The stacked vowel model architecture in particular offers promise to enhance such an approach.

Lay summary: AI analysis of multiple vowel recordings can improve classification of voice pathologies compared with models using a single sustained vowel and offer a strategy to enhance AI-driven screening of voice disorders.

Level of evidence: 3.

Keywords: artificial intelligence; deep learning; voice classification; voice disorders; voice pathology.