High speed surface defects detection of mirrors is of great significance, for detecting the quality of the mirrors on-site, and ultimately for monitoring the operating states of laser systems. The speeds of conventional proposals are relatively low as they utilize mechanically scanning methods or two-dimensional charge-coupled devices. Here, we propose a high speed surface detection method based on ultrafast single-pixel imaging, which consists of a spatial Fourier optical module for frequency-space mapping and a dispersive Fourier transform module for frequency-time mapping. An optical grating is utilized to map the wideband spectrum of dissipative soliton into the spatial domain under far-field diffraction, where the mirror is inspected. Dispersive Fourier transform is used to map the surface-defects-coded spectral information into the temporal domain, then recorded by a high speed single-pixel detector. The detection system permits continuous single-shot spectra measurement with a frame rate equivalent to the pulse repetition rate (8.4 MHz). We extract amplitude defects by demodulating light intensity, and obtain phase defects by demodulating the interference spectrum with a Mach-Zehnder interferometer structure. Experimental results show that the damaged mirror with a two-dimensional width of 10 × 13 mm can be obtained with a spatial resolution of 90 µm. The obtained phase accuracy after Hilbert transformation is 0.00217 rad, corresponding to a depth resolution of 51 nm. This scheme can find promising applications for surface defects detection of large aperture mirrors, and real-time monitoring of laser systems with high energy.