The design of a high finesse optical cavity made from two prism retroreflectors is fully described. Optical beam propagation calculations to determine the specification of prism angles and relative dimensions, the size of the astigmatic TEM00 beam as it propagates in the cavity, and the sensitivity of the optic axis to changes in prism alignment and fabrication errors are presented. The effects of material dispersion are also quantified for three different materials: fused silica, calcium fluoride, and barium fluoride. The predictions made are found to be in good agreement with experimental results obtained from prisms we had made from fused silica. Prisms made of CaF2 and BaF2 are predicted to be useful for applications in the UV and mid-IR spectral regions, respectively.