Exact geometric calibration of optical devices like projectors or cameras is the basis for utilizing them in quantitative metrological applications. The common state-of-the-art photogrammetric pinhole-imaging-based models with supplemental polynomial corrections fail in the presence of nonsymmetric or high-spatial-frequency distortions and in describing caustics efficiently. These problems are solved by our vision ray calibration (VRC), which is proposed in this paper. The VRC takes an optical mapping system modeled as a black box and directly delivers corresponding vision rays for each mapped pixel. The underlying model, the calibration process, and examples are visualized and reviewed, demonstrating the potential of the VRC.