The de Haas-van Alphen effect (dHvAE), describing oscillations of the magnetization as a function of magnetic field, is commonly assumed to be a definite sign for the presence of a Fermi surface (FS). Indeed, the effect forms the basis of a well-established experimental procedure for accurately measuring FS topology and geometry of metallic systems, with parameters commonly extracted by fitting to the Lifshitz-Kosevich (LK) theory based on Fermi liquid theory. Here we show that, in contrast to this canonical situation, there can be quantum oscillations even for band insulators of certain types. We provide simple analytic formulas describing the temperature dependence of the quantum oscillations in this setting, showing strong deviations from LK theory. We draw connections to recent experiments and discuss how our results can be used in future experiments to accurately determine, e.g., hybridization gaps in heavy-fermion systems.