The strong need in materials and biological science has prompted the development of high-speed quantitative phase imaging. However, for phase retrieval applying digital micromirror devices (DMDs), the accuracy of the retrieved phase will be disturbed by the DMD-induced aberrations. Here, we propose a phase retrieval method based on measuring and correcting errors caused by phase non-uniformity of the device. Using only four binary amplitude masks and corresponding diffraction intensities, the proposed method achieves rapid convergence and high-quality reconstruction. The experiments prove the practical feasibility for general samples and the effective improvement of the retrieved phase accuracy.