Develop an ADR prediction system of Chinese herbal injections containing Panax notoginseng saponin: a nested case-control study using machine learning

Abstract

Objective: This study aimed to develop an adverse drug reactions (ADR) antecedent prediction system using machine learning algorithms to provide the reference for security usage of Chinese herbal injections containing Panax notoginseng saponin in clinical practice.

Design: A nested case-control study.

Setting: National Center for ADR Monitoring and the Electronic Medical Record (EMR) system.

Participants: All patients were from five medical institutions in Sichuan Province from January 2010 to December 2018.

Main outcomes/measures: Data of patients with ADR who used Chinese herbal injections containing Panax notoginseng saponin were collected from the National Center for ADR Monitoring. A nested case-control study was used to randomly match patients without ADR from the EMR system by the ratio of 1:4. Eighteen machine learning algorithms were applied for the development of ADR prediction models. Area under curve (AUC), accuracy, precision, recall rate and F1 value were used to evaluate the predictive performance of the model. An ADR prediction system was established by the best model selected from the 1080 models.

Results: A total of 530 patients from five medical institutions were included, and 1080 ADR prediction models were developed. Among these models, the AUC of the best capable one was 0.9141 and the accuracy was 0.8947. According to the best model, a prediction system, which can provide early identification of patients at risk for the ADR of Panax notoginseng saponin, has been established.

Conclusion: The prediction system developed based on the machine learning model in this study had good predictive performance and potential clinical application.

Keywords: adverse events; herbal medicine; toxicity.