Magnetization dynamics and field dependent magnetization of different devices based on 25-30 nm thick Permalloy (Py) films: such as single Py layers (Py/MgO; Py/CoFeB/Al2O3) and Py inserted as a magnetic layer in magnetic tunnel junctions (Py/CoFe/Al2O3/CoFe; Py/CoFeB/Al2O3/CoFe; Py/MgO/Fe) have been extensively studied within a temperature range between 300 K down to 5 K. The dynamic response was investigated in the linear regime measuring the ferromagnetic resonance response of the Py layers using broadband vector network analyzer technique. Both the static and the dynamic properties suggest the possible presence of a thermally induced spin reorientation transition in the Py interface at temperatures around 60 K in all the samples investigated. It seems, however, that the details of the interface between Py and the hardening ferromagnet/insulator structure, the atomic structure of Py layers (amorphous vs. textured) as well as the presence of dipolar coupling through the insulating barrier in the magnetic tunnel junction structures could strongly influence this low temperature reorientation transition. Our conclusions are indirectly supported by structural characterization of the samples by means of X-Ray diffraction and high resolution transmission electron microscopy techniques. Micromagnetic simulations indicate the possibility of strongly enhanced surface anisotropy in thin Py films over CoFe or CoFeB underlayers. Comparison of the simulations with experimental results also shows that the thermally-induced spin reorientation transition could be influenced by the presence of strong disorder at the surface.