Landauer's principle states that it costs at least kBTln2 of work to reset one bit in the presence of a heat bath at temperature T. The bound of kBTln2 is achieved in the unphysical infinite-time limit. Here we ask what is possible if one is restricted to finite-time protocols. We prove analytically that it is possible to reset a bit with a work cost close to kBTln2 in a finite time. We construct an explicit protocol that achieves this, which involves thermalizing and changing the system's Hamiltonian so as to avoid quantum coherences. Using concepts and techniques pertaining to single-shot statistical mechanics, we furthermore prove that the heat dissipated is exponentially close to the minimal amount possible not just on average, but guaranteed with high confidence in every run. Moreover, we exploit the protocol to design a quantum heat engine that works near the Carnot efficiency in finite time.