In this work, we show how Gibbs or thermal states appear dynamically in closed quantum many-body systems, building on the program of dynamical typicality. We introduce a novel perturbation theorem for physically relevant weak system-bath couplings that is applicable even in the thermodynamic limit. We identify conditions under which thermalization happens and discuss the underlying physics. Based on these results, we also present a fully general quantum algorithm for preparing Gibbs states on a quantum computer with a certified runtime and error bound. This complements quantum Metropolis algorithms, which are expected to be efficient but have no known runtime estimates and only work for local Hamiltonians.