A high-slope-efficiency single-frequency (SF) ytterbium-doped fiber laser, based on a Sagnac loop mirror filter (LMF), was demonstrated. It combined a simple linear cavity with a Sagnac LMF that acted as a narrow-bandwidth filter to select the longitudinal modes. And we introduced a polarization controller to restrain the spatial hole burning effect in the linear cavity. The system could operate at a stable SF oscillating at 1064 nm with the obtained maximum output power of 32 mW. The slope efficiency was found to be primarily dependent on the reflectivity of the fiber Bragg grating. The slope efficiency of multi-longitudinal modes was higher than 45%, and the highest slope efficiency of the single longitudinal mode we achieved was 33.8%. The power stability and spectrum stability were <2% and <0.1%, respectively, and the signal-to-noise ratio measured was around 60 dB.