We consider how the energy cost of bit reset scales with the time duration of the protocol. Bit reset necessarily takes place in finite time, where there is an extra penalty on top of the quasistatic work cost derived by Landauer. This extra energy is dissipated as heat in the computer, inducing a fundamental limit on the speed of irreversible computers. We formulate a hardware-independent expression for this limit in the framework of stochastic processes. We derive a closed-form lower bound on the work penalty as a function of the time taken for the protocol and bit reset error. It holds for discrete as well as continuous systems, assuming only that the master equation respects detailed balance.