A Dataset for Temporal Semantic Segmentation Dedicated to Smart Mobility of Wheelchairs on Sidewalks

J Imaging. 2022 Aug 7;8(8):216. doi: 10.3390/jimaging8080216.

Abstract

In smart mobility, the semantic segmentation of images is an important task for a good understanding of the environment. In recent years, many studies have been made on this subject, in the field of Autonomous Vehicles on roads. Some image datasets are available for learning semantic segmentation models, leading to very good performance. However, for other types of autonomous mobile systems like Electric Wheelchairs (EW) on sidewalks, there is no specific dataset. Our contribution presented in this article is twofold: (1) the proposal of a new dataset of short sequences of exterior images of street scenes taken from viewpoints located on sidewalks, in a 3D virtual environment (CARLA); (2) a convolutional neural network (CNN) adapted for temporal processing and including additional techniques to improve its accuracy. Our dataset includes a smaller subset, made of image pairs taken from the same places in the maps of the virtual environment, but from different viewpoints: one located on the road and the other located on the sidewalk. This additional set is aimed at showing the importance of the viewpoint in the result of semantic segmentation.

Keywords: convolutional neural network; dataset; deep learning; semantic segmentation; smart mobility.

Grants and funding

The work presented in this paper is supported by the ADAPT (“Assistive Devices for empowering disAbled People through robotic Technologies”, http://adapt-project.com/index.php (accessed on 3 August 2022)) project which aims at contributing to the development of assisted and autonomous wheelchairs for people with reduced mobility. It is carried out within the INTERREG program, which is an European Territorial Cooperation Program aimed at funding high quality cooperation projects in the Channel border region between France and England. The Program is financed by the European Regional Development Fund (ERDF)).