Polarization modulation plays a key role in polarization-encoding quantum key distribution (QKD). Here, we report a new, to the best of our knowledge, polarization modulation scheme based on an inherently stable Sagnac interferometer. The presented scheme is free of polarization mode dispersion and calibration as well as insensitive to environmental influences. Successful experiments at a repetition frequency of 1.25 GHz have been conducted to demonstrate the feasibility and stability of the scheme. The measured average quantum bit-error rate of the four polarization states is as low as 0.27% for 80 consecutive minutes without any adjustment. This high-speed intrinsically stable polarization modulation can be widely applied to many polarization-encoding QKD systems, such as BB84, MDI, etc.