Smart scan of medical device displays to integrate with a mHealth application

Pedro Lobo; João L Vilaça; Helena Torres; Bruno Oliveira; Alberto Simões

doi:10.1016/j.heliyon.2023.e16297

Smart scan of medical device displays to integrate with a mHealth application

Heliyon. 2023 May 29;9(6):e16297. doi: 10.1016/j.heliyon.2023.e16297. eCollection 2023 Jun.

Authors

Pedro Lobo^{1

2}, João L Vilaça^{1

2}, Helena Torres^{1

2

3

4

5}, Bruno Oliveira^{1

2

3

4

5}, Alberto Simões^{1

2}

Affiliations

¹ 2AI, School of Technology, IPCA, Barcelos, Portugal.
² LASI - Associate Laboratory of Intelligent Systems, Guimarães, Portugal.
³ Algoritmi Center, School of Engineering, University of Minho, Guimarães, Portugal.
⁴ Life and Health Sciences Research Institute (ICVS), School of Medicine, University of Minho, Braga, Portugal.
⁵ ICVS/3B's -PT Government Associate Laboratory, Braga/Guimarães, Portugal.

Abstract

Background: The daily monitoring of the physiological parameters is essential for monitoring health condition and to prevent health problems. This is possible due to the democratization of numerous types of medical devices and promoted by the interconnection between these and smartphones. Nevertheless, medical devices that connect to smartphones are typically limited to manufacturers applications.

Objectives: This paper proposes an intelligent scanning system to simplify the collection of data displayed on different medical devices screens, recognizing the values, and optionally integrating them, through open protocols, with centralized databases.

Methods: To develop this system, a dataset comprising 1614 images of medical devices was created, obtained from manufacturer catalogs, photographs and other public datasets. Then, three object detector algorithms (yolov3, Single-Shot Detector [SSD] 320 × 320 and SSD 640 × 640) were trained to detect digits and acronyms/units of measurements presented by medical devices. These models were tested under 3 different conditions to detect digits and acronyms/units as a single object (single label), digits and acronyms/units as independent objects (two labels), and digits and acronyms/units individually (fifteen labels). Models trained for single and two labels were completed with a convolutional neural network (CNN) to identify the detected objects. To group the recognized digits, a condition-tree based strategy on density spatial clustering was used.

Results: The most promising approach was the use of the SSD 640 × 640 for fifteen labels.

Conclusion: Lastly, as future work, it is intended to convert this system to a mobile environment to accelerate and streamline the process of inserting data into mobile health (mhealth) applications.

Keywords: AI (Artificial intelligence); Consumer health information; Data aggregation; Data collection; Medical devices; OCR (Optical character recognition); e-health; mHealth.