Stereoscopic imagers are widely used in machine vision for three-dimensional (3D) visualization as well as in non-destructive testing for quantitative characterization of cracks, delamination and other defects. Measurement capability in these systems is provided by a proper combination of the optical parameters and data processing techniques. Conventional approach to their design consists of two sequential stages: optical system design and optimization of calibration and image processing algorithms. Such two-stage procedure often complicates both the hardware and the software, and results in a time-ineffective design procedure and cost-ineffective solution. We demonstrate a more effective approach and propose to estimate errors of 3D measurements at the early optical design stage. We show that computer simulation using optical design software allows not only optimizing optical parameters of the imager but also choosing the most effective mathematical model of the system and the equipment necessary for calibration. We tested the proposed approach on the design of miniature prism-based stereoscopic system and analyzed the impact of various factors (aberrations, tolerances, etc.) as on the image quality, so on the quality of calibration and 3D measurements accuracy. The proposed joint design approach may be highly effective for various measurement systems and applications when both optical parameters and image processing algorithms are not defined in advance and are necessary to be optimized.