We propose a novel three-dimensional (3D) imaging technique by terahertz (THz) waves. Specifically, we modulate the THz wave using diffusers to produce three different speckle-like illumination patterns. The object is raster scanned by the three illumination patterns to generate three raw images via the single-pixel detection method. Subsequently, we synthesize a complex field using the three raw images. Finally, the retrieved image is calculated using the phase correlation of the complex point spread function. The proposed imaging system is simple and highly cost-effective. Therefore, it is a promising technique that can be adopted for industrial inspection and security screening.