Three-dimensional surface information acquisition of specular objects plays an important role in the fields of automobile industry, aerospace, cultural relic protection, intelligent robotics, equipment manufacturing, and so on. Most of the existing specular surface measurement methods are based on focused sinusoidal fringe patterns, so there are certain requirements for the range of the depth of field (DOF) of the camera on the focus position. However, for many specular surfaces with a large gradient, the tested objects may not always be in the DOF of the camera, so sinusoidal fringe patterns are defocused to be vulnerable to the noise. In this Letter, a new infrared phase measuring deflectometry (PMD) based on defocused binary fringe is proposed that combines a binary fringe defocusing technique and direct PMD. The measurement principle and the corresponding system calibration method are described. The feasibility and measurement accuracy of fringe defocus in specular measurement are studied in principle. The experimental results on several specular objects show that the proposed method can effectively measure specular surfaces out of the DOF of the camera.