Rich literature has examined the impact of the built environment on commuting distance. Linear models assume that the influence of the built environment is spatially homogeneous. However, given the spatial heterogeneity of urban space, conclusions might be different or even be contrary. The influence of the built environment might also be different by home and work locations. To explore the spatially heterogeneous effect of the built environment from both home-based and work-based perspectives, this study applied large-scale cellular cellphone data in Guangzhou, China. Commuting was measured by decay parameters of probabilistic distributions of commuting distances. Geographically weighted regression models were applied to examine the spatially heterogeneous effect, differentiated by home-based and work-based perspectives. Results confirmed that the impact of the built environment on commuting distance is spatially heterogeneous. The urban space is classified into clusters of central areas, inner suburbs, and outer suburbs. Results also revealed the double-edged effect of the built environment. Residential population, recreation facilities, and mixed development are residence-attractive factors that increase the home-based commuting distance and decrease the work-based commuting distance. Work population and transport facilities are work-attractive factors that decrease home-based commuting distance and increase work-based commuting distance. The results further provide evidence to support area-based policies in urban planning practice.