The deflectometry provides a powerful metrological technique enabling the high-precision testing of reflective surfaces with high dynamic range, such as aspheric and freeform surfaces. In the fringe-illumination deflectometry based on reverse-Hartmann-test configuration, the calibration of system geometry is required to achieve "null" testing. However, the system miscalibration can introduce a significant systematic error in the testing results. A general double-step calibration method, which is based on the low-order Zernike aberration optimization and high-order aberration separation, is proposed to separate and eliminate the geometrical error due to system miscalibration. Both the numerical simulation and experiments have been performed to validate the feasibility of the proposed calibration method. The proposed method provides a general way for the accurate calibration of system geometrical error, avoids the over-correction and is feasible for the testing of various complex freeform surfaces.