The effect of finite temperature on the optical properties of nanostructures has been a longstanding problem for their theoretical description and its omission presents serious limits on the validity of calculated spectra and radiative lifetimes. Most ab initio calculations have been carried out neglecting temperature effects altogether, although progress has been made recently. In the present work, the temperature dependence of the intrinsic radiative lifetimes of excited electron-hole pairs in Ge and Si nanocrystals due to classical temperature effects is calculated using ab initio molecular dynamics. Fully hydrogen-saturated Ge and Si nanocrystals without surface reconstructions show opposite behavior: the very short lifetimes in Ge increase with temperature, while the much longer ones in Si decrease. However, the temperature effect is found to be strongly dependent on the surface structure: surface reconstructions cause partial localization of the wave functions and override the difference between Si and Ge. As a consequence, the temperature dependence in reconstructed nanocrystals is strongly attenuated compared to the fully saturated nanocrystals. Our calculations are an important step towards predictive modeling of the optical properties of nanostructures.