We describe a three-dimensional microscopy technique based on spectral and frequency encoding. The method employs a wavelength-swept laser to illuminate a specimen with a spectrally-dispersed line focus that sweeps over the specimen in time. The spatial information along each spectral line is further mapped into different modulation frequencies. Spectrally-resolved detection and subsequent Fourier analysis of the back-scattered light from the specimen therefore enable high-speed, scanner-free imaging of the specimen with a single-element photodetector. High-contrast, three-dimensional imaging capability of this method is demonstrated by presenting images of various materials and biological specimens.