Assessing the electronic Bedside Paediatric Early Warning System: A simulation study on decision-making and usability

Jessica N Tomasi; Megan V Hamilton; Mark Fan; Sonia J Pinkney; Kristen L Middaugh; Christopher S Parshuram; Patricia L Trbovich

doi:10.1016/j.ijmedinf.2019.103969

Assessing the electronic Bedside Paediatric Early Warning System: A simulation study on decision-making and usability

Int J Med Inform. 2020 Jan:133:103969. doi: 10.1016/j.ijmedinf.2019.103969. Epub 2019 Sep 14.

Authors

Jessica N Tomasi¹, Megan V Hamilton², Mark Fan³, Sonia J Pinkney⁴, Kristen L Middaugh⁵, Christopher S Parshuram⁶, Patricia L Trbovich⁷

Affiliations

¹ HumanEra, Office of Research and Innovation, North York General Hospital, 4001 Leslie Street, LE-140, North York, ON, Canada. Electronic address: jessica.tomasi@mail.utoronto.ca.
² HumanEra, Office of Research and Innovation, North York General Hospital, 4001 Leslie Street, LE-140, North York, ON, Canada; Institute of Biomaterials and Biomedical Engineering, University of Toronto; 164 College St, Toronto, ON, Canada. Electronic address: meganv.hamilton@mail.utoronto.ca.
³ HumanEra, Office of Research and Innovation, North York General Hospital, 4001 Leslie Street, LE-140, North York, ON, Canada. Electronic address: mark.fan@nygh.on.ca.
⁴ Institute of Health Policy, Management, and Evaluation, University of Toronto; 155 College Street, Toronto, ON, Canada. Electronic address: sonia.pinkney@uhn.ca.
⁵ Centre for Safety Research, The Hospital for Sick Children; 555 University Avenue, Toronto, ON, Canada. Electronic address: kristen.middaugh@sickkids.ca.
⁶ Institute of Health Policy, Management, and Evaluation, University of Toronto; 155 College Street, Toronto, ON, Canada; Centre for Safety Research, The Hospital for Sick Children; 555 University Avenue, Toronto, ON, Canada. Electronic address: chris@sickkids.ca.
⁷ HumanEra, Office of Research and Innovation, North York General Hospital, 4001 Leslie Street, LE-140, North York, ON, Canada; Institute of Biomaterials and Biomedical Engineering, University of Toronto; 164 College St, Toronto, ON, Canada; Institute of Health Policy, Management, and Evaluation, University of Toronto; 155 College Street, Toronto, ON, Canada. Electronic address: patricia.trbovich@utoronto.ca.

PMID: 31765879
DOI: 10.1016/j.ijmedinf.2019.103969

Abstract

Background: The Bedside Paediatric Early Warning System (BedsidePEWS) is a clinical decision support tool designed to augment clinician expertise, objectively identify children at risk for clinical deterioration, and standardize and prioritize care to improve outcomes in community settings. Although the paper-based BedsidePEWS documentation record has been shown to improve clinicians' perception of their ability to detect deterioration and follow care recommendations, research is needed to asses this impact empirically. Furthermore, as hospitals progressively move toward electronic clinical systems, knowledge regarding the impact of BedsidePEWS' novel electronic interface on clinicians' performance and user experience is required.

Objectives: The primary objectives of this study were (1) to compare adherence to evidence-based care recommendations using a) electronic health record software, b) paper BedsidePEWS, and c) a novel electronic BedsidePEWS interface, and (2) to describe end-users' experiences of usability and opportunities for improvement of both paper and electronic BedsidePEWS.

Methods: Paediatric nurses participated in a repeated measures simulation study. Participants assessed simulated patients, documented patient data, and responded to a series of questions regarding follow-up care for each patient. Three patient types (i.e., stable, mild deterioration, severe deterioration) were assessed in each of three intervention conditions (i.e., electronic health record, paper BedsidePEWS, electronic BedsidePEWS). Following simulation scenarios, participants provided comments regarding the usability of the paper and electronic tools.

Results: Participants made 12.7% and 18.0% more appropriate care decisions with paper and electronic BedsidePEWS, respectively, than with the electronic health record intervention (p < 0.001). Accurate BedsidePEWS severity of illness score calculation was related to better adherence to evidence-based care recommendations (65%), compared to inaccurate calculation (55%), and electronic BedsidePEWS was associated with 15.7% fewer calculation errors than paper (p < 0.005). Electronic BedsidePEWS demonstrated usability benefits over its paper predecessor, including automatic score calculation and data plotting, and the potential to eliminate double charting, and participants expressed a preference for electronic BedsidePEWS in all aspects of the debrief questionnaire (p < 0.001).

Conclusions: BedsidePEWS in both paper and electronic formats significantly improved participants' ability to detect deterioration and follow care recommendations compared to electronic health record software. Furthermore, results suggest that electronic BedsidePEWS would afford improved patient care in excess of the paper-based original and further contribute to the standardization, prioritization, and improvement of care in community settings.

Keywords: Electronic early warning system; Evidence-based care; Paediatrics; Simulation; Usability.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Adult
Decision Making*
Decision Support Systems, Clinical
Electronic Health Records
Female
Humans
Male
Middle Aged
Research Design
Software
Surveys and Questionnaires
Young Adult