Optical Microphone-Based Speech Reconstruction System With Deep Learning for Individuals With Hearing Loss

IEEE Trans Biomed Eng. 2023 Dec;70(12):3330-3341. doi: 10.1109/TBME.2023.3285437. Epub 2023 Nov 21.

Abstract

Objective: Although many speech enhancement (SE) algorithms have been proposed to promote speech perception in hearing-impaired patients, the conventional SE approaches that perform well under quiet and/or stationary noises fail under nonstationary noises and/or when the speaker is at a considerable distance. Therefore, the objective of this study is to overcome the limitations of the conventional speech enhancement approaches.

Method: This study proposes a speaker-closed deep learning-based SE method together with an optical microphone to acquire and enhance the speech of a target speaker.

Results: The objective evaluation scores achieved by the proposed method outperformed the baseline methods by a margin of 0.21-0.27 and 0.34-0.64 in speech quality (HASQI) and speech comprehension/intelligibility (HASPI), respectively, for seven typical hearing loss types.

Conclusion: The results suggest that the proposed method can enhance speech perception by cutting off noise from speech signals and mitigating interference caused by distance.

Significance: The results of this study show a potential way that can help improve the listening experience in enhancing speech quality and speech comprehension/intelligibility for hearing-impaired people.

MeSH terms

  • Cochlear Implants*
  • Deep Learning*
  • Hearing Aids*
  • Hearing Loss*
  • Humans
  • Speech Intelligibility
  • Speech Perception*