We demonstrate a high peak power mode-locked Yb:CaF2 oscillator pumped by a single-mode laser diode. The laser operated in hybrid Kerr-lens and SESAM mode-locked regime. Its performance was optimized by varying the output coupler ratio. Pulses as short as 65 fs were generated with 0.4% transmission. Employing 5% output coupler enabled generation of 77 fs pulses with 46 kW of peak power (262 mW of average output power). We believe that such high peak powers can open a way to practical applications of single-mode diode-pumped ultrafast ytterbium lasers.