SPNet: Structure preserving network for depth completion

PLoS One. 2023 Jan 24;18(1):e0280886. doi: 10.1371/journal.pone.0280886. eCollection 2023.

Abstract

Depth completion aims to predict a dense depth map from a sparse one. Benefiting from the powerful ability of convolutional neural networks, recent depth completion methods have achieved remarkable performance. However, it is still a challenging problem to well preserve accurate depth structures, such as tiny structures and object boundaries. To tackle this problem, we propose a structure preserving network (SPNet) in this paper. Firstly, an efficient multi-scale gradient extractor (MSGE) is proposed to extract useful multi-scale gradient images, which contain rich structural information that is helpful in recovering accurate depth. The MSGE is constructed based on the proposed semi-fixed depthwise separable convolution. Meanwhile, we adopt a stable gradient MAE loss (LGMAE) to provide additional depth gradient constrain for better structure reconstruction. Moreover, a multi-level feature fusion module (MFFM) is proposed to adaptively fuse the spatial details from low-level encoder and the semantic information from high-level decoder, which will incorporate more structural details into the depth modality. As demonstrated by experiments on NYUv2 and KITTI datasets, our method outperforms some state-of-the-art methods in terms of both quantitative and quantitative evaluations.

Publication types

  • Research Support, Non-U.S. Gov't

MeSH terms

  • Neural Networks, Computer*
  • Semantics

Grants and funding

This work was supported by the National Natural Science Foundation of China (Grant No. 61901392), the Department of Science and Technology of Sichuan Province (Grant No. 2021YJ0109). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.