Fruit-CoV: An efficient vision-based framework for speedy detection and diagnosis of SARS-CoV-2 infections through recorded cough sounds

Long H Nguyen; Nhat Truong Pham; Van Huong Do; Liu Tai Nguyen; Thanh Tin Nguyen; Hai Nguyen; Ngoc Duy Nguyen; Thanh Thi Nguyen; Sy Dzung Nguyen; Asim Bhatti; Chee Peng Lim

doi:10.1016/j.eswa.2022.119212

Fruit-CoV: An efficient vision-based framework for speedy detection and diagnosis of SARS-CoV-2 infections through recorded cough sounds

Expert Syst Appl. 2023 Mar 1:213:119212. doi: 10.1016/j.eswa.2022.119212. Epub 2022 Nov 7.

Authors

Long H Nguyen¹, Nhat Truong Pham^{2

3}, Van Huong Do⁴, Liu Tai Nguyen¹, Thanh Tin Nguyen⁵, Hai Nguyen⁶, Ngoc Duy Nguyen⁷, Thanh Thi Nguyen⁸, Sy Dzung Nguyen^{9

10}, Asim Bhatti⁷, Chee Peng Lim⁷

Affiliations

¹ Faculty of Information Technology, Ton Duc Thang University, Ho Chi Minh City, Vietnam.
² Division of Computational Mechatronics, Institute for Computational Science, Ton Duc Thang University, Ho Chi Minh City, Vietnam.
³ Faculty of Electrical and Electronics Engineering, Ton Duc Thang University, Ho Chi Minh City, Vietnam.
⁴ ASICLAND, Suwon, South Korea.
⁵ Human Computer Interaction Lab, Sejong University, Seoul, South Korea.
⁶ Khoury College of Computer Sciences, Northeastern University, Boston, USA.
⁷ Institute for Intelligent Systems Research and Innovation, Deakin University, Victoria, Australia.
⁸ School of Information Technology, Deakin University, Victoria, Australia.
⁹ Laboratory for Computational Mechatronics, Institute for Computational Science and Artificial Intelligence, Van Lang University, Ho Chi Minh City, Vietnam.
¹⁰ Faculty of Mechanical-Electrical and Computer Engineering, Van Lang University, Ho Chi Minh City, Vietnam.

Abstract

COVID-19 is an infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). This deadly virus has spread worldwide, leading to a global pandemic since March 2020. A recent variant of SARS-CoV-2 named Delta is intractably contagious and responsible for more than four million deaths globally. Therefore, developing an efficient self-testing service for SARS-CoV-2 at home is vital. In this study, a two-stage vision-based framework, namely Fruit-CoV, is introduced for detecting SARS-CoV-2 infections through recorded cough sounds. Specifically, audio signals are converted into Log-Mel spectrograms, and the EfficientNet-V2 network is used to extract their visual features in the first stage. In the second stage, 14 convolutional layers extracted from the large-scale Pretrained Audio Neural Networks for audio pattern recognition (PANNs) and the Wavegram-Log-Mel-CNN are employed to aggregate feature representations of the Log-Mel spectrograms and the waveform. Finally, the combined features are used to train a binary classifier. In this study, a dataset provided by the AICovidVN 115M Challenge is employed for evaluation. It includes 7,371 recorded cough sounds collected throughout Vietnam, India, and Switzerland. Experimental results indicate that the proposed model achieves an Area Under the Receiver Operating Characteristic Curve (AUC) score of 92.8% and ranks first on the final leaderboard of the AICovidVN 115M Challenge. Our code is publicly available.

Keywords: COVID-19; Deep learning; Delta variant; EfficientNet; Log-Mel spectrogram; Machine vision; Neural network; PANNs; Recorded cough sounds; Remote detection; SARS-CoV-2 infections; Self-testing service; Sound classification; Speedy detection; Wavegram.