Purpose: We present and validate a rule-based algorithm for the detection of moderate to severe liver-related immune-related adverse events (irAEs) in a real-world patient cohort. The algorithm can be applied to studies of irAEs in large data sets.
Methods: We developed a set of criteria to define hepatic irAEs. The criteria include: the temporality of elevated laboratory measurements in the first 2-14 weeks of immune checkpoint inhibitor (ICI) treatment, steroid intervention within 2 weeks of the onset of elevated laboratory measurements, and intervention with a duration of at least 2 weeks. These criteria are based on the kinetics of patients who experienced moderate to severe hepatotoxicity (Common Terminology Criteria for Adverse Events grades 2-4). We applied these criteria to a retrospective cohort of 682 patients diagnosed with hepatocellular carcinoma and treated with ICI. All patients were required to have baseline laboratory measurements before and after the initiation of ICI.
Results: A set of 63 equally sampled patients were reviewed by two blinded, clinical adjudicators. Disagreements were reviewed and consensus was taken to be the ground truth. Of these, 25 patients with irAEs were identified, 16 were determined to be hepatic irAEs, 36 patients were nonadverse events, and two patients were of indeterminant status. Reviewers agreed in 44 of 63 patients, including 19 patients with irAEs (0.70 concordance, Fleiss' kappa: 0.43). By comparison, the algorithm achieved a sensitivity and specificity of identifying hepatic irAEs of 0.63 and 0.81, respectively, with a test efficiency (percent correctly classified) of 0.78 and outcome-weighted F1 score of 0.74.
Conclusion: The algorithm achieves greater concordance with the ground truth than either individual clinical adjudicator for the detection of irAEs.