Development and validation of an automated delirium risk assessment system (Auto-DelRAS) implemented in the electronic health record system

Abstract

Background: A key component of the delirium management is prevention and early detection.

Objective: To develop an automated delirium risk assessment system (Auto-DelRAS) that automatically alerts health care providers of an intensive care unit (ICU) patient's delirium risk based only on data collected in an electronic health record (EHR) system, and to evaluate the clinical validity of this system.

Design: Cohort and system development designs were used.

Setting: Medical and surgical ICUs in two university hospitals in Seoul, Korea.

Participants: A total of 3284 patients for the development of Auto-DelRAS, 325 for external validation, 694 for validation after clinical applications.

Methods: The 4211 data items were extracted from the EHR system and delirium was measured using CAM-ICU (Confusion Assessment Method for Intensive Care Unit). The potential predictors were selected and a logistic regression model was established to create a delirium risk scoring algorithm to construct the Auto-DelRAS. The Auto-DelRAS was evaluated at three months and one year after its application to clinical practice to establish the predictive validity of the system.

Results: Eleven predictors were finally included in the logistic regression model. The results of the Auto-DelRAS risk assessment were shown as high/moderate/low risk on a Kardex screen. The predictive validity, analyzed after the clinical application of Auto-DelRAS after one year, showed a sensitivity of 0.88, specificity of 0.72, positive predictive value of 0.53, negative predictive value of 0.94, and a Youden index of 0.59.

Conclusions: A relatively high level of predictive validity was maintained with the Auto-DelRAS system, even one year after it was applied to clinical practice.

Keywords: Auto-DelRAS; Automated delirium risk assessment; Delirium; Electronic health record; Intensive care unit.