Polarization remote sensing technology expands the dimensions of the target and enriches its basic information over traditional remote sensing methods. During the imaging process, polarization imaging changes the polarization information of the target by the modulation of the optical system, affecting the detection accuracy. We term the modulation of the polarization state of light by an optical system as polarization aberration, and we found that a lens group combined with mirrors is beneficial in suppressing polarization aberrations. This study analyzed the principles of suppression and the polarization aberration of the optical system before and after suppression. Simulation results show that the diattenuation's average value is reduced by 51.1% and the retardance's average value is reduced by 26.3% after suppression. The corrected polarization cross-coupled energy is reduced by 73.18% in the central field of view and by 69.80% in the fringe field of view. Adding a lens group also effectively suppresses traditional aberrations and expands the field of view.