Assessing the reliability of convolutional neural network (CNN)-based CT imaging techniques is critical for reliable deployment in practice. Some evaluation methods exist but require full access to target CNN architecture and training data, something not available for proprietary or commercial algorithms. Moreover, there is a lack of systematic evaluation methods. To address these issues, we propose a patient-specific uncertainty and bias quantification (UNIQ) method that integrates knowledge distillation and Bayesian deep learning. Knowledge distillation creates a transparent CNN ("Student CNN") to approximate the target non-transparent CNN ("Teacher CNN"). Student CNN is built as a Bayesian-deep-learning-based probabilistic CNN that, for each input, always generates statistical distribution of the corresponding outputs, and characterizes predictive mean and two major uncertainties - data and model uncertainty. UNIQ was evaluated using a low-dose CT denoising task. Patient and phantom scans with routine-dose and synthetic quarter-dose were used to create training, validation, and testing sets. To demonstrate, Unet and Resnet were used as backbones of Teacher CNN and Student CNN respectively and were trained using independent training sets. Student Resnet was qualitatively and quantitatively evaluated. The pixel-wise predictive mean, data uncertainty, and model uncertainty from Student Resnet were very similar to the counterparts from Teacher Unet (mean-absolute-error: predictive mean 1.5HU, data uncertainty 1.8HU, model uncertainty 1.3HU; mean 2D correlation coefficient: total uncertainty 0.90, data uncertainty 0.86, model uncertainty 0.83). The proposed UNIQ can potentially systematically characterize the reliability of non-transparent CNN models used in CT.
Keywords: Bayesian neural network; Deep learning; bias; knowledge distillation; uncertainty; x-ray CT.