We present a comprehensive and systematic investigation of the fundamental physical limitations of Faraday isolation performance at high average powers that are due to thermally induced birefringence. First, the operation of various Faraday isolator designs by use of arbitrary orientation of cubic magneto-optic crystals is studied theoretically. It is shown that, for different Faraday isolator designs, different crystal orientations can optimize the isolation ratio. Second, thermo-optic and photoelastic constants for terbium gallium garnet crystals grown by different manufacturers were measured. Measurements of self-induced depolarization are made for various orientations of crystallographic axes. The measurements are in good agreement with our theoretical predictions. Based on our results, it is possible to select a crystal orientation that optimizes isolation performance at high average powers, resulting in a 5-dB enhancement over nonoptimized orientations.