Symptom-based scoring technique by machine learning to predict COVID-19: a validation study

Abstract

Background: Coronavirus disease 2019 (COVID-19) surges, such as that which occurred when omicron variants emerged, may overwhelm healthcare systems. To function properly, such systems should balance detection and workloads by improving referrals using simple yet precise and sensitive diagnostic predictions. A symptom-based scoring system was developed using machine learning for the general population, but no validation has occurred in healthcare settings. We aimed to validate a COVID-19 scoring system using self-reported symptoms, including loss of smell and taste as major indicators.

Methods: A cross-sectional study was conducted to evaluate medical records of patients admitted to Dr. Sardjito Hospital, Yogyakarta, Indonesia, from March 2020 to December 2021. Outcomes were defined by a reverse-transcription polymerase chain reaction (RT-PCR). We compared the symptom-based scoring system, as the index test, with antigen tests, antibody tests, and clinical judgements by primary care physicians. To validate use of the index test to improve referral, we evaluated positive predictive value (PPV) and sensitivity.

Results: After clinical judgement with a PPV of 61% (n = 327/530, 95% confidence interval [CI]: 60% to 62%), confirmation with the index test resulted in the highest PPV of 85% (n = 30/35, 95% CI: 83% to 87%) but the lowest sensitivity (n = 30/180, 17%, 95% CI: 15% to 19%). If this confirmation was defined by either positive predictive scoring or antigen tests, the PPV was 92% (n = 55/60, 95% CI: 90% to 94%). Meanwhile, the sensitivity was 88% (n = 55/62, 95% CI: 87% to 89%), which was higher than that when using only antigen tests (n = 29/41, 71%, 95% CI: 69% to 73%).

Conclusions: The symptom-based COVID-19 predictive score was validated in healthcare settings for its precision and sensitivity. However, an impact study is needed to confirm if this can balance detection and workload for the next COVID-19 surge.

Keywords: COVID-19; Clinical prediction rules; Hospital referral; Machine learning; Validation study.