So far, the operation of ultrafast bulk laser oscillators based on Yb-doped gain materials and directly emitting few-cycle pulses have been restricted to low optical-to-optical efficiencies and average output powers of only a few milliwatt. This performance limitation can be attributed to the commonly-applied standard collinear pumping scheme in which the optical pump is transmitted through a dichroic mirror whose spectral transmission and dispersion properties severely perturb the oscillating pulse when its optical spectrum extends towards the pump wavelength. In this study, we report on a novel pumping scheme relying on cross polarization that overcomes this challenge. In our concept, the pump transmitting mirror is highly transmissive for the pump light in p-polarization, while it is highly reflective for the laser light in s-polarization over a broad wavelength range, even covering the pump wavelength and beyond. In contrast to a standard thin-film polarizer featuring similar polarization dependent properties, it provides a low and flat dispersion profile over a broad spectral range for the s-polarization. Implementing this pumping scheme in a soft-aperture Kerr-lens mode-locked bulk laser oscillator based on the gain material Yb:CALGO, we achieve clean 22-fs soliton pulses at 729 mW of average output power and an optical-to-optical efficiency of 25%. In a second configuration optimized for the highest average output power, we demonstrate a high optical-to-optical efficiency of 36.6%, which was obtained for 31-fs pulses at 1.63 W of average output power. In a third configuration we experimentally confirm the limiting effect of a dichroic mirror commonly used in the standard collinear pumping scheme. All the results presented here and obtained in the first and second configuration generate pulses with a center wavelength ranging from 1030 nm to 1056 nm, well within the spectral region of high gain cross sections of Yb:CALGO. While this initial demonstration was realized using a commercial diffraction-limited fiber laser as pump source, the pump geometry appears also well suited for pumping with laser diodes coupled into multimode fibers. This novel approach opens up new opportunities for compact and cost-efficient high-power few-cycle bulk laser oscillators based on Yb-doped gain materials and can be applied to any gain material with small quantum defect.