We analyze and suppress the magnetic noise response in optical rotation detection system (ORDS) in atomic magnetometers in this study. Because of the imperfections of the optical elements, the probe light is actually elliptically polarized in ORDS, which can polarize the atom ensemble and cause the responses to the three-axis magnetic noise. We theoretically analyze the frequency responses to the magnetic noise, and prove that the responses are closely associated with the DC magnetic field. The values of the DC magnetic fields are calculated with special frequency points, called 'break points', in the transverse responses. We reveal the relationships between the DC magnetic field and the sensitivities of ORDS, and effectively suppress the magnetic noise responses with the residual magnetic field compensation. Finally, the sensitivity of ORDS is improved by approximately two times at 10-20 Hz.