When a binocular vision sensor (BVS) is installed in a narrow space, traditional calibration methods are limited as all targets should be placed in more than three different positions. To solve this problem, an on-site calibration method based on the phase-shift algorithm is proposed in our paper. Intrinsic parameters of these two cameras should be first calibrated offline. Series of phase-shift patterns are projected onto any one target with known three-dimensional information to determine the relationship between two cameras. The target utilized in our proposed method can be selected arbitrarily, which is suitable to achieve the on-site calibration of BVS, especially in industrial vibration environments. Experiments are conducted to validate the effectiveness and robustness of our proposed method.