A learning approach with incomplete pixel-level labels for deep neural networks

Learning with incomplete labels in Neural Networks has been actively investigated these last years. Among different kinds of incomplete labels, we investigate incomplete pixel-level labels which are tackled in many concrete problems. One of the challenges for incomplete pixel-level labels is the missing information at local-level. Most of the current researches with incomplete labels in Neural Network focus on the incompleteness of global labels, only a few works focus on the incompleteness of local labels. To deal with the local incompleteness, we propose a learning approach which uses two dynamic weighted maps in parallel: one for object pixels and another one for background pixels. The two maps are integrated into the loss function of the target Neural Networks, to optimize the model by the present labels and to minimize the damage of the missing labels. We validate our approach on the speech balloon extraction problem in comic book images. Our approach uses the output of a balloon extraction algorithm as incomplete labels. The results are comparable with the state of the art supervised approach with manual labels. The results are very promising because our method does not require any manual labels. In addition, we apply our method to the medical image segmentation task to confirm the generalization of our approach.