We show that graphene deposited on a substrate has a non-negligible density of atomic scale defects. This is evidenced by a previously unnoticed D peak in the Raman spectra with intensity of ∼1% with respect to the G peak. We evaluated the effect of such impurities on electron transport by mimicking them with hydrogen adsorbates and measuring the induced changes in both mobility and Raman intensity. If the intervalley scatterers responsible for the D peak are monovalent, their concentration is sufficient to account for the limited mobilities currently achievable in graphene on a substrate.