Personalizing renal replacement therapy initiation in the intensive care unit: a reinforcement learning-based strategy with external validation on the AKIKI randomized controlled trials

Abstract

Objective: The timely initiation of renal replacement therapy (RRT) for acute kidney injury (AKI) requires sequential decision-making tailored to individuals' evolving characteristics. To learn and validate optimal strategies for RRT initiation, we used reinforcement learning on clinical data from routine care and randomized controlled trials.

Materials and methods: We used the MIMIC-III database for development and AKIKI trials for validation. Participants were adult ICU patients with severe AKI receiving mechanical ventilation or catecholamine infusion. We used a doubly robust estimator to learn when to start RRT after the occurrence of severe AKI for three days in a row. We developed a "crude strategy" maximizing the population-level hospital-free days at day 60 (HFD60) and a "stringent strategy" recommending RRT when there is significant evidence of benefit for an individual. For validation, we evaluated the causal effects of implementing our learned strategies versus following current best practices on HFD60.

Results: We included 3748 patients in the development set and 1068 in the validation set. Through external validation, the crude and stringent strategies yielded an average difference of 13.7 [95% CI -5.3 to 35.7] and 14.9 [95% CI -3.2 to 39.2] HFD60, respectively, compared to current best practices. The stringent strategy led to initiating RRT within 3 days in 14% of patients versus 38% under best practices.

Discussion: Implementing our strategies could improve the average number of days that ICU patients spend alive and outside the hospital while sparing RRT for many.

Conclusion: We developed and validated a practical and interpretable dynamic decision support system for RRT initiation in the ICU.

Keywords: acute kidney injury; causal inference; personalized medicine; reinforcement learning; renal replacement therapy.