Photoacoustic spectroscopy has been shown to be a promising tool for non-invasive blood glucose monitoring. However, the repeatability of such a method is susceptible to changes in skin condition, which is dependent on hand washing and drying due to the high absorption of infrared excitation light to the skin secretion products or water. In this paper, we present a method to meet the challenges of mid-infrared photoacoustic spectroscopy for non-invasive glucose monitoring. By obtaining the microscopic spatial information of skin during the spectroscopy measurement, the skin region where the infrared spectra is insensitive to skin condition can be locally selected, which enables reliable prediction of the blood glucose level from the photoacoustic spectroscopy signals. Our raster-scan imaging showed that the skin condition for in vivo spectroscopic glucose monitoring had significant inhomogeneities and large variability in the probing area where the signal was acquired. However, the selective localization of the probing led to a reduction in the effects of variability due to the skin secretion product. Looking forward, this technology has broader applications not only in continuous glucose monitoring for diabetic patient care, but in forensic science, the diagnosis of malfunctioning sweat pores, and the discrimination of tumors extracted via biopsy.