A voice-based digital assistant for intelligent prompting of evidence-based practices during ICU rounds

Andrew J King; Derek C Angus; Gregory F Cooper; Danielle L Mowery; Jennifer B Seaman; Kelly M Potter; Leigh A Bukowski; Ali Al-Khafaji; Scott R Gunn; Jeremy M Kahn

doi:10.1016/j.jbi.2023.104483

A voice-based digital assistant for intelligent prompting of evidence-based practices during ICU rounds

J Biomed Inform. 2023 Oct:146:104483. doi: 10.1016/j.jbi.2023.104483. Epub 2023 Aug 30.

Authors

Affiliations

¹ Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Scaife Hall Suite 600, 3550 Terrace Street, Pittsburgh, PA 15261, USA. Electronic address: andrew.king@pitt.edu.
² Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Scaife Hall Suite 600, 3550 Terrace Street, Pittsburgh, PA 15261, USA. Electronic address: angusdc@upmc.edu.
³ Department of Biomedical Informatics, University of Pittsburgh School of Medicine, Offices at Baum 4th Floor, 5607 Baum Blvd, Pittsburgh, PA 15206, USA. Electronic address: gfc@pitt.edu.
⁴ Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania School of Medicine, Blockley Hall 8th Floor, 423 Guardian Drive, Philadelphia, PA 19104, USA. Electronic address: dlmowery@pennmedicine.upenn.edu.
⁵ Department of Acute & Tertiary Care, University of Pittsburgh School of Nursing, 336 Victoria Building, 3500 Victoria Street, Pittsburgh, PA 15261, USA. Electronic address: jbs31@pitt.edu.
⁶ Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Scaife Hall Suite 600, 3550 Terrace Street, Pittsburgh, PA 15261, USA. Electronic address: kelly.potter@pitt.edu.
⁷ Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Scaife Hall Suite 600, 3550 Terrace Street, Pittsburgh, PA 15261, USA. Electronic address: lab108@pitt.edu.
⁸ Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Scaife Hall Suite 600, 3550 Terrace Street, Pittsburgh, PA 15261, USA. Electronic address: alkhafajia2@upmc.edu.
⁹ Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Scaife Hall Suite 600, 3550 Terrace Street, Pittsburgh, PA 15261, USA. Electronic address: gunnsr@ccm.upmc.edu.
¹⁰ Department of Critical Care Medicine, University of Pittsburgh School of Medicine, Scaife Hall Suite 600, 3550 Terrace Street, Pittsburgh, PA 15261, USA. Electronic address: jeremykahn@pitt.edu.

PMID: 37657712
PMCID: PMC10591951 (available on 2024-10-01)
DOI: 10.1016/j.jbi.2023.104483

Abstract

Objective: To evaluate the technical feasibility and potential value of a digital assistant that prompts intensive care unit (ICU) rounding teams to use evidence-based practices based on analysis of their real-time discussions.

Methods: We evaluated a novel voice-based digital assistant which audio records and processes the ICU care team's rounding discussions to determine which evidence-based practices are applicable to the patient but have yet to be addressed by the team. The system would then prompt the team to consider indicated but not yet delivered practices, thereby reducing cognitive burden compared to traditional rigid rounding checklists. In a retrospective analysis, we applied automatic transcription, natural language processing, and a rule-based expert system to generate personalized prompts for each patient in 106 audio-recorded ICU rounding discussions. To assess technical feasibility, we compared the system's prompts to those created by experienced critical care nurses who directly observed rounds. To assess potential value, we also compared the system's prompts to a hypothetical paper checklist containing all evidence-based practices.

Results: The positive predictive value, negative predictive value, true positive rate, and true negative rate of the system's prompts were 0.45 ± 0.06, 0.83 ± 0.04, 0.68 ± 0.07, and 0.66 ± 0.04, respectively. If implemented in lieu of a paper checklist, the system would generate 56% fewer prompts per patient, with 50%±17% greater precision.

Conclusion: A voice-based digital assistant can reduce prompts per patient compared to traditional approaches for improving evidence uptake on ICU rounds. Additional work is needed to evaluate field performance and team acceptance.

Keywords: Artificial intelligence; Checklist; Evidence-based practice; Implementation science; Intensive care units.

Abstract

Grants and funding