A low-noise fiber frequency comb is demonstrated to improve the frequency accuracy and linewidth by suppressing the phase noise caused by the nonlinear self-phase modulation as well as the amplified spontaneous emission within the Er-doped fiber amplifier. The linewidth of the carrier-envelop- offset signal measures less than 1.9 mHz and the frequency stability well follows the reference Rb clock. This achievement will facilitate the use of the fiber frequency comb for industrial applications to precision near-infrared spectroscopy, frequency calibration, optical clocks and length metrology.