Insulating granular aluminum is one of the proto-typical disordered insulators whose low temperature electrical conductance exhibits ubiquitous non-equilibrium phenomena. These include slow responses to temperature or gate voltage changes, characteristic field effect anomalies and ageing phenomena typical of a glass. In this system the influence of temperature on the glassy dynamics has remained elusive. A similar situation was met in insulating indium oxide and it was concluded that in high carrier density Anderson insulators, electronic slow relaxations essentially proceed via activationless processes. In this work we experimentally demonstrate that thermal effects do play a role and that the slow dynamics in granular aluminum is subject to thermal activation. We show how its signatures can be revealed and activation energy distributions can be extracted, providing a promising grasp on the nature of the microscopic mechanism at work in glassy Anderson insulators. We explain why some of the experimental protocols previously used in the literature fail to reveal thermal activation in these systems. Our results and analyses call for a reassessment of the emblematic case of indium oxide, and question the existence of purely activationless dynamics in any of the systems studied so far.