Stereo cameras have been widely used for three-dimensional (3D) photogrammetry, and stereo calibration is a crucial process to estimate the intrinsic and extrinsic parameters. This paper proposes a stereo calibration method with absolute phase target by using horizontal and vertical phase-shifting fringes. The one-to-one mapping from the world points to the image points that can be recovered by referring to the absolute phase and then used to calibrate the stereo cameras. Compared with traditional methods that only use feature points within the overlapping field-of-view (FOV), the proposed method can use all feature points within the overlapping and non-overlapping FOVs. Besides, since phase is more robust against camera defocusing than intensity, the target images can be captured regardless of the depth-of-field (DOF). With the advantages of whole-field capability and defocusing tolerability, the target placement becomes very flexible. Both simulations and experiment results demonstrate the robustness and accuracy of the proposed method.