Precise calibration of a binocular vision system is the foundation of binocular vision measurement. In this paper, we propose a highly precise and robust binocular camera calibration method, which is devoted to minimize the error between the geometric relation of 3D reconstructed feature points and the ground truth, such as adjacent distance error, collinear error, and right-angle error. In addition, the reprojection error and epipolar are introduced to satisfy the homography relation and epipolar geometry theory better. We optimize all intrinsic parameters, extrinsic parameters, and distortion parameters to minimize the objective function, which is the sum of a series of nonlinear least squares terms. Levenberg-Marquardt iterative algorithm is used to find the optimal solution of the camera parameters. To test the precision and robustness of the proposed method, both actual measurement experiments and Gauss noise-adding experiments are carried out. The experimental results show that compared with the other two calibration methods in the contrast experiment, the distance measurement error, collinear error, and right-angle error are reduced dramatically. It is noticeable that in Gauss noise-adding experiments, the calibration parameters estimated by the proposed method are more stable.