In this work, a narrow-linewidth polarization-maintaining (PM) all-fiber amplifier with near-diffraction-limited beam quality and record output power is presented. First, a 4.45-kW PM fiber amplifier with a 3-dB linewidth of 0.08 nm and root mean square (rms) linewidth of 0.22 nm is achieved based on optimized phase modulation. However, the sideband of the spectrum broadens significantly during the amplification process, which is mainly caused by the additional intensity variation of the injected signal. Meanwhile, an up to 5.04-kW linearly polarized fiber laser with a relatively stable spectral bandwidth is achieved by effectively suppressing spectral broadening. At the maximum output power, the rms linewidth is 0.2 nm, the beam quality factor M2 is less than 1.3, the polarization extinction ratio (PER) is 16.5 dB, and the signal-to-noise ratio (SNR) is approximately 53 dB. The further power scaling of the amplifier is mainly limited by the pump power. To the best of our knowledge, this is the maximum output power of a narrow linewidth linearly polarized fiber amplifier to date.