In this work we report on the influence of nitrogen ambient thermal effects on the performance of Pt/Al2O3/Si3N4/SiO2/Si memory capacitors. Two post deposition annealing (PDA) furnace steps were employed, at 850 and 1050 °C both for 15 min. The alumina films were deposited by atomic layer deposition using TMA/H2O at 250 °C. The structural characteristics of the stacks were evaluated by transmission electron microscopy and x-ray reflectivity measurements. The memory performance of the stacks was evaluated by write/erase and erase/write measurements, endurance and retention testing. It was found that in as-deposited state the Al2O3 layer is defective resulting in strong leakage currents, controlled by deep defects states. Thus, this behavior inhibits the memory functionality of the stacks. PDA crystallizes and condenses the Al2O3 transforming the layer from amorphous to polycrystalline. During this transformation the Al2O3 electrical quality improves greatly indicating that a significant number of these deep defects have been removed during annealing. Physical reasoning implies that the most plausible origin of these deep defects is hydrogen. However, the polycrystalline Al2O3 films showed inferior retention characteristics which are attributed to grain boundary related shallow defects. The findings of this work could pave the way for more efficient annealing schemes, in which an important factor is the time interval for hydrogen out-diffusion from the Al2O3 layer.