Extreme heat loads on optics, in particular the final pulse compression gratings, are a major hurdle to overcome in the ongoing push towards high average power (kW) and high repetition rate (kHz) operation of terawatt-class Ti:sapphire lasers. Multilayer dielectric (MLD) diffraction gratings have been suggested as a potential alternative to traditionally gold-coated compressor gratings, which are plagued by high energy absorption in the top gold layer. However, to support the required bandwidth (and ultimately the desired pulse duration) with MLD gratings, the gratings have to be operated in an out-of-plane geometry near the Littrow angle. Here, we report on the design of an MLD-based out-of-plane test compressor and a matching custom stretcher. We present a full characterization of the MLD compressor, focusing on its spectral transmission and the significance of laser pulse polarization in the out-of-plane geometry. To demonstrate compression of 40 μJ pulses centered at 800 nm wavelength to 26 fs pulse duration, we use the compressor with an MLD and gold grating configuration, and fully characterize the compressed pulses. Extrapolating our results indicates that MLD-grating-based out-of-plane compressors can support near-transform-limited pulses with sub-30 fs duration and good quality, demonstrating the viability of this concept for kW-level ultrafast Ti:sapphire laser systems.