We demonstrate a novel amplification regime in a counter-pumped, relatively long (2 meters), large mode area, highly Yb-doped and polarization-maintaining tapered fiber, which offers a high peak power directly from the amplifier. The main feature of this regime is that the amplifying signal propagates through a thin part of the tapered fiber without amplification and experiences an extremely high gain in the thick part of the tapered fiber, where most of the pump power is absorbed. In this regime, we have demonstrated 8 ps pulse amplification to a peak power of up to 0.76 MW, which is limited by appearance of stimulated Raman scattering. In the same regime, 28 ps chirped pulses are amplified to a peak power of 0.35 MW directly from the amplifier and then compressed with 70% efficiency to 315 ± 10 fs, corresponding to an estimated peak power of 22 MW.