Saliency guided data augmentation strategy for maximally utilizing an object's visual information

PLoS One. 2022 Oct 13;17(10):e0274767. doi: 10.1371/journal.pone.0274767. eCollection 2022.

Abstract

Among the various types of data augmentation strategies, the mixup-based approach has been particularly studied. However, in existing mixup-based approaches, object loss and label mismatching can occur if random patches are utilized when constructing augmented images, and additionally, patches that do not contain objects might be included, which degrades performance. In this paper, we propose a novel augmentation method that mixes patches in a non-overlapping manner after they are extracted from the salient regions in an image. The suggested method can make effective use of object characteristics, because the constructed image consists only of visually important regions and is robust to noise. Since the patches do not occlude each other, the semantically meaningful information in the salient regions can be fully utilized. Additionally, our method is more robust to adversarial attack than the conventional augmentation method. In the experimental results, when Wide ResNet was trained on the public datasets, CIFAR-10, CIFAR-100 and STL-10, the top-1 accuracy was 97.26%, 83.99% and 82.40% respectively, which surpasses other augmentation methods.

Grants and funding

This research was supported by the Chung-Ang University Research Scholarship Grants in 2021, and the Ministry of Culture, Sports and Tourism and Korea Creative Content Agency (Project Number: R2020040186). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.