Holography has paved the way for phase imaging in a variety of wide-field techniques, including electron, X-ray and optical microscopy. In scanning optical microscopy, however, the serial fashion of image acquisition seems to challenge a direct implementation of traditional holography. Here we introduce synthetic optical holography (SOH) for quantitative phase-resolved imaging in scanning optical microscopy. It uniquely combines fast phase imaging, technical simplicity and simultaneous operation at visible and infrared frequencies with a single reference arm. We demonstrate SOH with a scattering-type scanning near-field optical microscope (s-SNOM) where it enables reliable quantitative phase-resolved near-field imaging with unprecedented speed. We apply these capabilities to nanoscale, non-invasive and rapid screening of grain boundaries in CVD-grown graphene, by recording 65 kilopixel near-field images in 26 s and 2.3 megapixel images in 13 min. Beyond s-SNOM, the SOH concept could boost the implementation of holography in other scanning imaging applications such as confocal microscopy.