Multicenter validation of PIM3 and PIM2 in Brazilian pediatric intensive care units

Abstract

Objective: To validate the PIM3 score in Brazilian PICUs and compare its performance with the PIM2.

Methods: Observational, retrospective, multicenter study, including patients younger than 16 years old admitted consecutively from October 2013 to September 2019. We assessed the Standardized Mortality Ratio (SMR), the discrimination capability (using the area under the receiver operating characteristic curve - AUROC), and the calibration. To assess the calibration, we used the calibration belt, which is a curve that represents the correlation of predicted and observed values and their 95% Confidence Interval (CI) through all the risk ranges. We also analyzed the performance of both scores in three periods: 2013-2015, 2015-2017, and 2017-2019.

Results: 41,541 patients from 22 PICUs were included. Most patients aged less than 24 months (58.4%) and were admitted for medical conditions (88.6%) (respiratory conditions = 53.8%). Invasive mechanical ventilation was used in 5.8%. The median PICU length of stay was three days (IQR, 2-5), and the observed mortality was 1.8% (763 deaths). The predicted mortality by PIM3 was 1.8% (SMR 1.00; 95% CI 0.94-1.08) and by PIM2 was 2.1% (SMR 0.90; 95% CI 0.83-0.96). Both scores had good discrimination (PIM3 AUROC = 0.88 and PIM2 AUROC = 0.89). In calibration analysis, both scores overestimated mortality in the 0%-3% risk range, PIM3 tended to underestimate mortality in medium-risk patients (9%-46% risk range), and PIM2 also overestimated mortality in high-risk patients (70%-100% mortality risk).

Conclusions: Both scores had a good discrimination ability but poor calibration in different ranges, which deteriorated over time in the population studied.

Keywords: PIM2; PIM3; benchmarking; intensive care units; mortality; pediatric; validation study.