Using Facial Landmark Detection on Thermal Images as a Novel Prognostic Tool for Emergency Departments

Abstract

Introduction: Emergency departments (ED) at hospitals sometimes experience unexpected deterioration in patients that were assessed to be in a stable condition upon arrival. Odense University Hospital (OUH) has conducted a retrospective study to investigate the possibilities of prognostic tools that can detect these unexpected deterioration cases at an earlier stage. The study suggests that the temperature difference (gradient) between the core and the peripheral body parts can be used to detect these cases. The temperature between the patient's inner canthus (core temperature) and the tip of the nose (peripheral temperature) can be measured with a thermal camera. Based on the temperature measurement from a thermal image, a gradient value can be calculated, which can be used as an early indicator of potential deterioration.

Problem: The lack of a tool to automatically calculate the gradient has prevented the ED at OUH in conducting a comprehensive prospective study on early indicators of patients at risk of deterioration. The current manual way of doing facial landmark detection on thermal images is too time consuming and not feasible as part of the daily workflow at the ED, where nurses have to triage patients within a few minutes.

Objective: The objective of this study was to automate the process of calculating the gradient by developing a handheld prognostic tool that can be used by nurses for automatically performing facial landmark detection on thermal images of patients as they arrive at the ED.

Methods: A systematic literature review has been conducted to investigate previous studies that have been done for applying computer vision methods on thermal images. Several meetings, interviews and field studies have been conducted with the ED at OUH in order to understand their workflow, formulate and prioritize requirements and co-design the prognostic tool.

Results: The study resulted in a novel Android app that can capture a thermal image of a patient's face with a thermal camera attached to a smartphone. Within a few seconds, the app then automatically calculates the gradient to be used in the triage process. The developed tool is the first of its kind using facial landmark detection on thermal images for calculating a gradient that can serve as a novel prognostic indicator for ED patients.

Keywords: computer vision; machine learning; prognosis; thermal imaging; triage.