Incremental benefit of computer-aided detection when used as a second and concurrent reader of CT colonographic data: multiobserver study

Abstract

Purpose: To quantify the changes in reader performance levels, if any, during interpretation of computed tomographic (CT) colonographic data when a computer-aided detection (CAD) system is used as a second or concurrent reader.

Materials and methods: After institutional review board approval was obtained, 16 experienced radiologists searched for polyps in 112 patients, 56 of whom had 132 polyps. Each case was interpreted on three separate occasions by using an unassisted (without CAD), second-read CAD, or concurrent CAD reading paradigm. The reading paradigm and case order were randomized, with a minimal interval of 1 month between consecutive interpretations. The readers' findings were compared with the reference-truth interpretation. The mean per-patient sensitivity and mean per-patient specificity with CAD were compared with those achieved with unassisted reading. An increase in per-patient sensitivity was considered to be clinically more important than an equivalent decrease in specificity.

Results: The mean per-patient sensitivity for identification of patients with polyps of any size increased significantly with use of second-read CAD (mean increase, 7.0%; 95% confidence interval [CI]: 4.0%, 9.8%) and concurrent CAD (mean increase, 4.5%; 95% CI: 0.8%, 8.2%). The mean per-patient specificity did not decrease significantly with use of second-read CAD (mean decrease, -2.5%; 95% CI: -5.2%, 0.1%) or concurrent CAD (mean decrease, -2.2%; 95% CI: -4.6%, 0.2%). With analysis restricted to patients with polyps 6 mm or larger, the benefit in sensitivity with second-read CAD remained (mean increase, 7.1%; 95% CI: 3.0%, 11.1%), whereas the increase with concurrent CAD was not significant (mean increase, 4.2%; 95% CI: -0.5%, 8.9%). Use of second-read CAD significantly increased the per-polyp sensitivity for polyps 6 mm or larger (mean increase, 9.0%; 98.3% CI: 4.9%, 12.8%) and polyps 5 mm or smaller (mean increase, 5.9%; 98.3% CI: 3.2%, 9.1%), but use of concurrent CAD increased the per-polyp sensitivity for only those polyps 5 mm or smaller (mean increase, 4.8%; 98.3% CI: 2.2%, 7.9%).

Conclusion: Use of second-read CAD significantly improves readers' per-patient and per-polyp detection. Concurrent CAD is less effective.

Supplemental material: http://radiology.rsna.org/lookup/suppl/doi:10.1148/radiol.10100354/-/DC1.