Background: For virtual skincare using a touch feedback interface, reconstructing a 3D skin tactile surface from a mobile skin image is imperative for a dermatologist to palpate the skin surface that presents tactile characteristics of the subcutaneous tissues. However, the precise tactile reconstruction from a single view image is a challenging research problem due to varying illumination conditions.
Methods: In this study, a deep learning-based tactile reconstruction scheme is proposed to restore tactile properties from light distortion and reconstruct the 3D tactile surface from a mobile skin image. Our method consists of light distortion removal using deep learning, cGAN, and 3D tactile surface generation based on image gradients.
Results: The proposed method was tested by conducting two evaluation experiments in terms of removing light distortion and reconstructing 3D skin tactile surface in comparison with other well-known methods. The results demonstrated that our method outperforms existing other methods in both illumination-free image restoration and 3D surface reconstruction.
Conclusion: The proposed method is a promising approach in that tactile property distorted by illuminations can be completely restored using deep learning with a smaller training set and the precise reconstruction of 3D skin tactile surface can be achieved to be ready for a remotely touchable interface for virtual skincare applications.
Keywords: 3D tactile surface reconstruction; deep learning; image restoration; light distortion removal; skin tactile reconstruction; virtual palpation; virtual skincare.
© 2021 John Wiley & Sons A/S. Published by John Wiley & Sons Ltd.