Benefits of Independent Double Reading in Digital Mammography: A Theoretical Evaluation of All Possible Pairing Methodologies

Abstract

Rationale and objectives: To establish the efficacy of pairing readers randomly and evaluate the merits of developing optimal pairing methodologies.

Materials and methods: Sensitivity, specificity, and proportion correct were computed for three different case sets that were independently read by 16 radiologists. Performance of radiologists as single readers was compared to expected double reading performance. We theoretically evaluated all possible pairing methodologies. Bootstrap resampling methods were used for statistical analyses.

Results: Significant improvements in expected performance for double versus single reading (ie, delta performance) were shown for all performance measures and case-sets (p ≤ .003), with overall delta performance across all theoretically possible pairing schemes (n = 10,395) ranging between .05 and .08. Delta performance for the 20 best pairing schemes was significant (p < .001) and ranged between .07 and .10. Delta performance for 20 random pairing schemes was also significant (p ≤ .003) and ranged between .05 and .08. Delta performance for the 20 worst pairing schemes ranged between .03 and .06, reaching significance in delta proportion correct (p ≤ .021) for all three case-sets and in delta specificity for two case-sets (p ≤ .033) but not for a third case-set (p = .131), and not reaching significance in delta sensitivity for any of the three case-sets (.098 ≥ p ≥ .067).

Conclusion: Significant benefits accrue from double reading, and while random reader pairing achieves most double reading benefits, a strategic pairing approach may maximize the benefits of double reading.

Keywords: Breast cancer; Digital mammography; Double reading; Observer variation; Radiologists.