Here, we report the presence of defect-related states with magnetic degrees of freedom in crystals of LaAlO3 and several other rare-earth based perovskite oxides using inelastic light scattering (Raman spectroscopy) at low temperatures in applied magnetic fields of up to 9 T. Some of these states are at about 140 meV above the valence band maximum while others are mid-gap states at about 2.3 eV. No magnetic impurity could be detected in LaAlO3 by Proton-Induced X-ray Emission Spectroscopy. We, therefore, attribute the angular momentum-like states in LaAlO3 to cationic/anionic vacancies or anti-site defects. Comparison with the other rare earth perovskites leads to the empirical rule that the magnetic-field-sensitive transitions require planes of heavy elements (e.g. lanthanum) and oxygen without any other light cations in the same plane. These magnetic degrees of freedom in rare earth perovskites with useful dielectric properties may be tunable by appropriate defect engineering for magneto-optic applications.