Object-stable unsupervised dual contrastive learning image-to-image translation with query-selected attention and convolutional block attention module

PLoS One. 2023 Nov 6;18(11):e0293885. doi: 10.1371/journal.pone.0293885. eCollection 2023.

Abstract

Recently, contrastive learning has gained popularity in the field of unsupervised image-to-image (I2I) translation. In a previous study, a query-selected attention (QS-Attn) module, which employed an attention matrix with a probability distribution, was used to maximize the mutual information between the source and translated images. This module selected significant queries using an entropy metric computed from the attention matrix. However, it often selected many queries with equal significance measures, leading to an excessive focus on the background. In this study, we proposed a dual-learning framework with QS-Attn and convolutional block attention module (CBAM) called object-stable dual contrastive learning generative adversarial network (OS-DCLGAN). In this paper, we utilize a CBAM, which learns what and where to emphasize or suppress, thereby refining intermediate features effectively. This CBAM was integrated before the QS-Attn module to capture significant domain information for I2I translation tasks. The proposed framework outperformed recently introduced approaches in various I2I translation tasks, showing its effectiveness and versatility. The code is available at https://github.com/RedPotatoChip/OSUDL.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Entropy
  • Learning*
  • Probability

Grants and funding

This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (RS-2023-00280241), and the fund of research promotion program, Gyeongsang National University, 2022, and "Regional Innovation Strategy (RIS)" through the NRF funded by the Ministry of Education(MOE)(2021RIS-003). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.