Domain-specific classification-pretrained fully convolutional network encoders for skin lesion segmentation

Abstract

Background and objective: Fully convolutional neural networks have been shown to perform well for automated skin lesion segmentation on digital dermatoscopic images. Our concept is that transferring encoder weights from a network trained on a classification task on images of the same domain may contain useful information for segmentation.

Methods: We trained a fully convolutional network where ResNet34 layers are reused as encoding layers of a U-Net style architecture. We entered the encoding layers i) with He uniform ("random") initialization, ii) pretrained ImageNet weights, or iii) after fine-tuning ResNet34 for skin lesion classification. After transferring the layers to the fully convolutional network architecture we trained for a binary segmentation task using official ISIC 2017 challenge data.

Results: Pretraining of ResNet34-layers with either ImageNet or fine-tuning for skin lesion classification achieved a higher Jaccard than random initialization (0.763 and 0.768 vs 0.740) on the ISIC 2017 test-set. This improved performance warrants further exploration on how to implement cross-task learning for skin lesion segmentation. In additional experiments we found that post-processing with fully connected conditional random fields consistently decreased Jaccard on ISIC 2017 test-set images despite reasonable visual results. Further exploration of the test-set revealed that conditional random field - post-processing decreased segmentation performance only if ground truth annotations consisted of simple shapes but increased it if shapes were complex.

Conclusions: Our findings suggest that domain specific pretraining of encoders can be helpful when there are only few ground truth masks available for segmentation training, but may not be of additional benefit to ImageNet pretraining given enough segmentation training data. Complexity of ground truth annotations have a large impact on segmentation metrics and should be taken into account in skin lesion segmentation research.

Keywords: Classification; Dermatoscopy; Fully convolutional networks; Segmentation.