We report on a combined photoluminescence imaging and atomic force microscopy study of single, isolated self-assembled InAs quantum dots. The motivation of this work is to determine an approach that allows to assess single quantum dots as candidates for quantum nanophotonic devices. By combining optical and scanning probe characterization techniques, we find that single quantum dots often appear in the vicinity of comparatively large topographic features. Despite this, the quantum dots generally do not exhibit significant differences in their non-resonantly pumped emission spectra in comparison to quantum dots appearing in defect-free regions, and this behavior is observed across multiple wafers produced in different growth chambers. Such large surface features are nevertheless a detriment to applications in which single quantum dots are embedded within nanofabricated photonic devices: they are likely to cause large spectral shifts in the wavelength of cavity modes designed to resonantly enhance the quantum dot emission, thereby resulting in a nominally perfectly-fabricated single quantum dot device failing to behave in accordance with design. We anticipate that the approach of screening quantum dots not only based on their optical properties, but also their surrounding surface topographies, will be necessary to improve the yield of single quantum dot nanophotonic devices.