The magneto optical Kerr effect (MOKE) is a widely used technique in magnetic domain imaging for its high surface sensitivity and external magnetic compatibility. Optimization of Kerr microscopy will improve the detecting sensitivity and provide high-quality domain images. In this work, we provide a method to optimize the polarizer azimuth in improving the signal-to-noise ratio (S/N) in longitudinal Kerr microscopy with the generalized magneto optical ellipsometry. Detailed analysis of the MOKE signal and the noise components are provided to study the optimum polarizer and analyzer azimuth combinations. Results show that, for a fixed polarizer angle 1°, the laser intensity noise and the shot noise, which vary with the input laser power, have a similar amplitude and decline with the analyzer azimuth increasing. When the analyzer is set at the extinction place, the Johnson noise plays a dominate role in the total noise. Then, the S/N values are calculated to find the optimum polarizer and analyzer azimuth. Results show that the optimum polarizer and analyzer azimuth combination for Permalloy is (18.35°, 68.35°) under an incident angle of 45°. After that, the S/N of 200 nm Permalloy at different analyzer angles with the polarizer azimuth set at 18.35° is measured to verify the validity of the simulation results. At last, the S/N at different incident angles is calculated. Results show that the optimum incident angle of 200 nm Permalloy film to improve the S/N is 70.35° under the polarizer and analyzer angles set at the optimal combinations (18.35°, 68.35°).