Does Training in LI-RADS Version 2018 Improve Readers' Agreement with the Expert Consensus and Inter-reader Agreement in MRI Interpretation?

Abstract

Background: The Liver Imaging Reporting and Data System (LI-RADS) was established for noninvasive diagnosis for hepatocellular carcinoma (HCC). However, whether training can improve readers' agreement with the expert consensus and inter-reader agreement for final categories is still unclear.

Purpose: To explore training effectiveness on readers' agreement with the expert consensus and inter-reader agreement.

Study type: Prospective.

Subjects: Seventy lesions in 61 patients at risk of HCC undergoing liver MRI; 20 visiting scholars.

Field strength/sequence: 1.5 T or 3 T, Dual-echo T₁ WI, Fast spin-echo T₂ WI, SE-EPI DWI, and Dynamic multiphase fast gradient-echo T₁ WI.

Assessment: Seventy lesions assigned LI-RADS categories of LR1-LR5, LR-M, and LR-TIV by three radiologists in consensus were randomly selected, with 10 cases for each category. The consensus opinion was the standard reference. The third radiologist delivered the training. Twenty readers reviewed images independently and assigned each an LI-RADS category both before and after the training.

Statistical tests: Accuracy, sensitivity, specificity, positive predictive value, negative predictive value, positive likelihood ratio, negative likelihood ratio, receiver operating characteristic (ROC) analysis, simple and weighted kappa statistics, and Fleiss kappa statistics.

Results: Before and after training: readers' AUC (areas under ROC) for LR-1-LR-5, LR-M, and LR-TIV were 0.898 vs. 0.913, 0.711 vs. 0.876, 0.747 vs. 0.860, 0.724 vs. 0.815, 0.844 vs. 0.895, 0.688 vs. 0.873, and 0.720 vs. 0.948, respectively, and all improved significantly (P < 0.05), except LR-1(P = 0.25). Inter-reader agreement between readers for LR-1-LR-5, LR-M, LR-TIV were 0.725 vs. 0.751, 0.325 vs. 0.607, 0.330 vs. 0.559, 0.284 vs. 0.488, 0.447 vs. 0.648, 0.229 vs. 0.589, and 0.362 vs. 0.852, respectively, and all increased significantly (P < 0.05). For training effectiveness on both AUC and inter-reader agreement, LR-TIV, LR-M, and LR-2 improved most, and LR-1 made the least.

Data conclusion: This study shows LI-RADS training could improve reader agreement with the expert consensus and inter-reader agreement for final categories.

Level of evidence: 2 TECHNICAL EFFICACY STAGE: 2.

Keywords: LI-RADS special training; inter-reader agreement; readers' agreement with the expert consensus.