We report an electro-optic modulator (EOM) with a wedged MgO: LiNbO3 as the modulation crystal to reduce the zero baseline drift (ZBD) of the Pound-Drever-Hall (PDH) error signal. When the input linear polarization is not along the modulation direction, the wedged design can separate the two orthogonal polarizations in space after the EOM and eliminate the interference between the carrier and the two orthogonal sidebands. Therefore, the residual amplitude modulation (RAM) of phase modulation process caused by the input polarization misalignment and the etalon effect can be significantly reduced. The noise power spectrum of phase-modulated light with wedged crystal EOM is suppressed from -24 to -69 dBm, which is much lower than that with conventional EOM. The peak-to-peak value of the ZBD of the PDH error signal is reduced effectively to +70/-50 ppm during the 10 h, which meets the requirements for stable squeezed light generation.