The optimal threshold for prompt clinical review: An external validation study of the national early warning score

Abstract

Aims and objectives: The aim of this study was to determine the optimal threshold for national early warning score in clinical practice.

Background: The national early warning score is an aggregate early warning score aiming to predict patient mortality. Studies validating national early warning score did not use standardised patient outcomes or did not always include clinical workload in their results. Since all patients with a positive national early warning score require a clinical workup, it is crucial to determine the optimal threshold to limit false-positive alerts.

Design: An external validation study using retrospectively collected data of patient admissions in six Belgian hospitals.

Methods: We adhered to the STARD guideline for reporting. Two sample groups were selected: the cross-sectional sample (admitted patients, 1 day every 4 months) and the serious adverse event sample (all patients with unexpected death, cardiac arrest and unplanned admission to the intensive care unit). The maximum registered national early warning score value was collected in both groups. Predictive values were used as estimates for clinical workload.

Results: We collected 1,523 in the cross-sectional sample and 390 patients in the serious adverse event sample. A national early warning score ≥5 had a predictive value of 6.8% and a negative predictive value of 99.5% to predict unexpected death, cardiac arrest with cardiopulmonary resuscitation or unplanned admission to intensive care (AUROC 0.841). The performance of national early warning score differed between outcome measures. Considering the predictive value, the optimal threshold for national early warning score is ≥5.

Conclusions: We validated national early warning score to be applied in general hospital wards and confirmed the optimal threshold (≥5).

Relevance to clinical practice: When a patient has a national early warning score <5, we may assume that in the next 24 hr this patient is less likely to die unexpectedly, receive cardiopulmonary resuscitation or be transferred to the ICU. Because of the significant number of false positives when national early warning score is ≥5, hospitals should create workable guidelines for clinical practice.

Keywords: burden; deterioration; early warning score; mortality; positive predictive value; sensitivity; specificity; validation; workload.