Optical anisotropy originates from crystalline structures with low symmetry and governs the polarization-dependent light propagation. Optical anisotropy is particularly important to lead halide perovskites that have been under intense investigation for optoelectronic and photonic applications, as this group of materials possesses rich structural phases that deviate from the high-symmetry cubic phase. Here we apply 2D optical Kerr effect spectroscopy to quantify the optical anisotropy in single-crystal methylammonium lead bromide (MAPbBr3). We determine the strong photon energy dependence of optical anisotropy near the band gap and show the dramatic change in optical anisotropy across phase transitions. We correlate the optical anisotropy with the structural anisotropy and demonstrate the tuning of optical anisotropy by alloyed CsxMA1-xPbBr3 perovskite crystals.