Visible light optical coherence tomography (OCT) has recently emerged in retinal imaging, with claims of micrometer-scale axial resolution and multi-color (sub-band) imaging. Here, we show that the large dispersion of optical glass and aqueous media, together with broad optical bandwidths often used in visible light OCT, compromises both of these claims. To rectify this, we introduce the notion of spatially dependent (i.e., depth and transverse position-dependent) dispersion. We use a novel sub-band, sub-image correlation algorithm to estimate spatially dependent dispersion in our 109 nm bandwidth visible light OCT mouse retinal imaging system centered at 587 nm. After carefully compensating spatially dependent dispersion, we achieve delineation of fine outer retinal bands in mouse strains of varying pigmentation. Spatially dependent dispersion correction is critical for broader bandwidths and shorter visible wavelengths.