An interpretable machine learning framework for measuring urban perceptions from panoramic street view images

Yunzhe Liu; Meixu Chen; Meihui Wang; Jing Huang; Fisher Thomas; Kazem Rahimi; Mohammad Mamouei

doi:10.1016/j.isci.2023.106132

An interpretable machine learning framework for measuring urban perceptions from panoramic street view images

iScience. 2023 Feb 3;26(3):106132. doi: 10.1016/j.isci.2023.106132. eCollection 2023 Mar 17.

Authors

Yunzhe Liu^{1

2}, Meixu Chen³, Meihui Wang⁴, Jing Huang^{1

5}, Fisher Thomas¹, Kazem Rahimi¹, Mohammad Mamouei¹

Affiliations

¹ Informal Cities, Oxford Martin School, University of Oxford, Oxford OX1 3BD, UK.
² MRC Centre for Environment and Health, School of Public Health, Imperial College London, London W2 1PG, UK.
³ Geographic Data Science Lab, Department of Geography and Planning, University of Liverpool, Liverpool L69 7ZT, UK.
⁴ SpaceTimeLab, Department of Civil, Environmental and Geomatic Engineering, University College London, London WC1E 6BT, UK.
⁵ Department of Occupational and Environmental Health Sciences, School of Public Health, Peking University, Beijing 100191, China.

Abstract

The proliferation of street view images (SVIs) and the constant advancements in deep learning techniques have enabled urban analysts to extract and evaluate urban perceptions from large-scale urban streetscapes. However, many existing analytical frameworks have been found to lack interpretability due to their end-to-end structure and "black-box" nature, thereby limiting their value as a planning support tool. In this context, we propose a five-step machine learning framework for extracting neighborhood-level urban perceptions from panoramic SVIs, specifically emphasizing feature and result interpretability. By utilizing the MIT Place Pulse data, the developed framework can systematically extract six dimensions of urban perceptions from the given panoramas, including perceptions of wealth, boredom, depression, beauty, safety, and liveliness. The practical utility of this framework is demonstrated through its deployment in Inner London, where it was used to visualize urban perceptions at the Output Area (OA) level and to verify against real-world crime rate.

Keywords: Artificial Intelligence; Environmental sciences.