Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural Networks

Hieu Nguyen; Yuzeng Wang; Zhaoyang Wang

doi:10.3390/s20133718

Single-Shot 3D Shape Reconstruction Using Structured Light and Deep Convolutional Neural Networks

Sensors (Basel). 2020 Jul 3;20(13):3718. doi: 10.3390/s20133718.

Authors

Hieu Nguyen^{1

2}, Yuzeng Wang³, Zhaoyang Wang¹

Affiliations

¹ Department of Mechanical Engineering, The Catholic University of America, Washington, DC 20064, USA.
² Neuroimaging Research Branch, National Institute on Drug Abuse, National Institutes of Health, Baltimore, MD 21224, USA.
³ School of Mechanical Engineering, Jinan University, Jinan 250022, China.

Abstract

Single-shot 3D imaging and shape reconstruction has seen a surge of interest due to the ever-increasing evolution in sensing technologies. In this paper, a robust single-shot 3D shape reconstruction technique integrating the structured light technique with the deep convolutional neural networks (CNNs) is proposed. The input of the technique is a single fringe-pattern image, and the output is the corresponding depth map for 3D shape reconstruction. The essential training and validation datasets with high-quality 3D ground-truth labels are prepared by using a multi-frequency fringe projection profilometry technique. Unlike the conventional 3D shape reconstruction methods which involve complex algorithms and intensive computation to determine phase distributions or pixel disparities as well as depth map, the proposed approach uses an end-to-end network architecture to directly carry out the transformation of a 2D image to its corresponding 3D depth map without extra processing. In the approach, three CNN-based models are adopted for comparison. Furthermore, an accurate structured-light-based 3D imaging dataset used in this paper is made publicly available. Experiments have been conducted to demonstrate the validity and robustness of the proposed technique. It is capable of satisfying various 3D shape reconstruction demands in scientific research and engineering applications.

Keywords: convolutional neural networks; deep machine learning; depth measurement; fringe projection; structured light; three-dimensional image acquisition; three-dimensional sensing; three-dimensional shape reconstruction.