This paper proposes an improved calibration method for a structured light system by using the random sample consensus (RANSAC) method with nonlinear optimization and an improved gray centroid method. The proposed method is composed of two steps: calibrating intrinsic and extrinsic parameters for the camera, exploiting the improved gray centroid method to extract the centerline, and fitting the structured light plane by the RANSAC approach with the three-dimensional (3D) points obtained from different positions. The error function caused by the extracted centerline is deduced based on the pixel error perturbation method. The error results of the 3D points are simulated and analyzed. An imaging system is built to realize the 3D imaging. The experimental results show that the calibration error is within 0.08 mm and the reconstruction error is less than 0.45 mm. Moreover, it performs better for the reconstruction of complex objects compared with traditional methods.