Phase measuring deflectometry is a powerful measuring method of complex optical surfaces that captures the reflected fringe images associated with a displaying screen and calculates the normal vectors of the surface under test (SUT) accordingly. The captured images are usually set conjugate to the SUT, which in turn makes the screen defocused. As a result, the blurring effect caused by the defocus and aberrations of the off-axis catadioptric imaging system can severely degrade the phases solved from the blurred images. In order to correct the phase errors, the space-variant point spread functions (PSFs) are modeled using a skew-normal function. The phase bias is estimated by forward convolution between the captured images and the PSF models. Demonstrated with a highly curved aspheric surface, the measurement accuracy can be improved by three times.