Background: Machine learning may assist with the identification of potentially inappropriate penicillin allergy labels. Strategies to improve the performance of existing models for this task include the use of additional training data, synthetic data and transfer learning.
Aims: The aims of this study were to investigate the use of additional training data and novel machine learning strategies, namely synthetic data and transfer learning, to improve the performance of penicillin adverse drug reaction (ADR) machine learning classification.
Methods: Machine learning natural language processing was applied to free-text penicillin ADR data extracted from a public health system electronic health record (EHR). The models were developed by training on various labelled data sets. ADR entries were split into training and testing data sets and used to develop and test a variety of machine learning models. The effect of training on additional data and synthetic data versus the use of transfer learning was analysed.
Results: Following the application of these techniques, the area under the receiver operator curve of best-performing models for the classification of penicillin allergy (vs intolerance) and high-risk allergy (vs low-risk allergy) improved to 0.984 (using the artificial neural network model) and 0.995 (with the transfer learning approach) respectively.
Conclusions: Machine learning models demonstrate high levels of accuracy in the classification and risk stratification of penicillin ADR labels using the reaction documented in the EHR. The model can be further optimised by incorporating additional training data and using transfer learning. Practical applications include automating case detection for penicillin allergy delabelling programmes.
Keywords: artificial intelligence; delabelling; natural language processing.
© 2024 Royal Australasian College of Physicians.