Development of a machine learning algorithm based on administrative claims data for identification of ED anaphylaxis patient visits

Ronna L Campbell; Mollie L Alpern; James T Li; John B Hagan; Megan Motosue; Aidan F Mullan; Lauren S Harper; Christine M Lohse; Molly M Jeffery

doi:10.1016/j.jacig.2022.09.002

Development of a machine learning algorithm based on administrative claims data for identification of ED anaphylaxis patient visits

J Allergy Clin Immunol Glob. 2022 Oct 17;2(1):61-68. doi: 10.1016/j.jacig.2022.09.002. eCollection 2023 Feb.

Authors

Ronna L Campbell¹, Mollie L Alpern², James T Li², John B Hagan², Megan Motosue³, Aidan F Mullan⁴, Lauren S Harper¹, Christine M Lohse⁴, Molly M Jeffery^{1

5}

Affiliations

¹ Department of Emergency Medicine, Mayo Clinic, Rochester.
² Division of Allergic Diseases, Mayo Clinic, Rochester.
³ Division of Allergy and Immunology, Kaiser Permanente Honolulu Clinic, Honolulu.
⁴ Division of Clinical Trials and Biostatistics, Mayo Clinic, Rochester.
⁵ Department of Health Sciences Research, Mayo Clinic, Rochester.

Abstract

Background: Epidemiologic studies of anaphylaxis commonly rely on International Classification of Diseases (ICD) codes to identify anaphylaxis cases, which may lead to suboptimal epidemiologic classification.

Objective: We sought to develop and assess the accuracy of a machine learning algorithm using ICD codes and other administrative data compared with ICD code-only algorithms to identify emergency department (ED) anaphylaxis visits.

Methods: We conducted a retrospective review of ED visits from January 2013 to September 2017. Potential ED anaphylaxis visits were identified using 3 methods: anaphylaxis ICD diagnostic codes (method 1), ICD symptom-based codes with or without a code indicating an allergic trigger (method 2), and ICD codes indicating a potential allergic reaction only (method 3). A machine learning algorithm was developed from administrative data, and test characteristics were compared with ICD code-only algorithms.

Results: A total of 699 of 2191 (31.9%) potential ED anaphylaxis visits were classified as anaphylaxis. The sensitivity and specificity of method 1 were 49.1% and 87.5%, respectively. Method 1 used in combination with method 2 resulted in a sensitivity of 53.9% and a specificity of 68.7%. Method 1 used in combination with method 3 resulted in a sensitivity of 98.4% and a specificity of 15.1%. The sensitivity and specificity of the machine learning algorithm were 87.3% and 79.1%, respectively.

Conclusions: ICD coding alone demonstrated poor sensitivity in identifying cases of anaphylaxis, with venom-related anaphylaxis missing 96% of cases. The machine learning algorithm resulted in a better balance of sensitivity and specificity and improves upon previous strategies to identify ED anaphylaxis visits.

Keywords: Anaphylaxis; emergency department; epidemiology; machine learning.