A prospective observational concordance study to evaluate computational model-driven clinical practice guidelines for Type 2 diabetes mellitus

Abstract

Background: Clinical Practice Guidelines (CPGs) provide healthcare professionals with performance and decision-making support during the treatment of patients. Sometimes, however, they are poorly implemented. The IDE₄ICDS platform was developed and validated with CPGs for type 2 diabetes mellitus (T2DM).

Objective: The main objective of this paper is to present the results of the clinical validation of the IDE₄ICDS platform in a real clinical environment at two health clinics in the Andalusian Public Health System (SSPA) in the southern Spanish region of Andalusia.

Methods: National and international knowledge sources on T2DM were selected and reviewed and used to define a diabetes CPG model on the IDE₄ICDS platform. Once the diabetes CPG was configured and deployed, it was validated. A total of 506 patients were identified as meeting the inclusion criteria, of whom 130 could be recruited and 89 attended the appointment.

Results: A concordance analysis was performed with the kappa value. Overall agreement between the recommendations provided by the system and those recorded in each patient's EHR was good (0.61 - 0.80) with a total kappa index of 0.701, leading to the conclusion that the system provided appropriate recommendations for each patient and was therefore well-functioning.

Conclusions: A series of possible improvements were identified based on the limitations for the recovery of variables related to the quality of these recolected variables, the detection of duplicate recommendations based on different input variables for the same patient, and clinical usability, such as the capacity to generate reports based on the recommendations generated. Nevertheless, the project resulted in the IDE₄ICDS platform: a Clinical Decision Support System (CDSS) capable of providing appropriate recommendations for improving the management and quality of patient care and optimizing health outcomes. The result of this validation is a safe and effective pathway for developing and adopting digital transformation at the regional scale of the use of biomedical knowledge in real healthcare.

Keywords: Clinical decision support system; Clinical practice guideline; Evidence-based medicine; Type 2 diabetes mellitus.