This paper presents a comprehensive study of the formation of surface relief gratings in a series of photoresponsive molecular glasses. Holographic experiments were performed on films of the azobenzene-containing molecular glasses. Seven relevant polarization configurations of the writing beams were systematically applied, and simultaneously the diffraction efficiency was monitored during the process of inscription. The temporal evolution of the diffraction efficiency can be precisely simulated with a model which takes both the surface relief and the phase grating in the volume into account. From the measured diffraction efficiencies, the modulation heights can be directly calculated and they were independently confirmed by atomic force microscopy. We found that all experimental results can be explained with the gradient force model, and we suggest that the grating heights generated with different writing polarizations can be ascribed to the varying strengths of the gradient force. For materials with different substituents at the azobenzene chromophore, the optical susceptibility at the writing laser wavelength and, therefore, the gradient force varies. By applying the most efficient polarization configuration in combination with the best material, we were able to reach modulation heights of up to 600 nm, which is a factor of 2 higher than modulations usually reported for azobenzene-containing polymers.