Validity of prognostic models of critical COVID-19 is variable. A systematic review with external validation

Abstract

Objectives: To identify prognostic models which estimate the risk of critical COVID-19 in hospitalized patients and to assess their validation properties.

Study design and setting: We conducted a systematic review in Medline (up to January 2021) of studies developing or updating a model that estimated the risk of critical COVID-19, defined as death, admission to intensive care unit, and/or use of mechanical ventilation during admission. Models were validated in two datasets with different backgrounds (HM [private Spanish hospital network], n = 1,753, and ICS [public Catalan health system], n = 1,104), by assessing discrimination (area under the curve [AUC]) and calibration (plots).

Results: We validated 18 prognostic models. Discrimination was good in nine of them (AUCs ≥ 80%) and higher in those predicting mortality (AUCs 65%-87%) than those predicting intensive care unit admission or a composite outcome (AUCs 53%-78%). Calibration was poor in all models providing outcome's probabilities and good in four models providing a point-based score. These four models used mortality as outcome and included age, oxygen saturation, and C-reactive protein among their predictors.

Conclusion: The validity of models predicting critical COVID-19 by using only routinely collected predictors is variable. Four models showed good discrimination and calibration when externally validated and are recommended for their use.

Keywords: COVID-19; Critical disease; Epidemiology; External validation; Intensive care unit; Prognostic models.