Cold atom experiments usually need a controllable and low-noise bias magnetic field to provide a quantization axis. Most labs need home-made stabilization of the field according to the actual setup, as commercially available power supply cannot directly satisfy their requirements. Here, by measuring the field fluctuations and active feedback modulating current supply of the applied magnetic field, we successfully demonstrate a field of 10.58 G with a stability to the level of 2.8 × 10-7 in a duration of 5 min. The root mean square noise is reduced to 0.05 mG, compared to the noise of 1.3 mG without stabilization. The coherence time of the magnetic-field sensitive transition between the rubidium ground states F=1,mF=-1 and 1,0, as measured by Rabi oscillation, is extended to 19.2 ms from the unstabilized value of 1.3 ms. This result is long enough for most experiments on quantum simulation and precision measurement. As our system has no passive magnetic shielding and additional compensation coils, it is highly simple and compact to provide the stable magnetic field and would be adapted to various applications with cold atoms.