Only a selected group of two-dimensional (2D) lead-halide perovskites shows a peculiar broad-band photoluminescence. Here we show that the structural distortions of the perovskite lattice can determine the defectivity of the material by modulating the defect formation energies. By selecting and comparing two archetype systems, namely, (NBT)2PbI4 and (EDBE)PbI4 perovskites (NBT = n-butylammonium and EDBE = 2,2-(ethylenedioxy)bis(ethylammonium)), we find that only the latter, subject to larger deformation of the Pb-X bond length and X-Pb-X bond angles, sees the formation of VF color centers whose radiative decay ultimately leads to broadened PL. These findings highlight the importance of structural engineering to control the optoelectronic properties of this class of soft materials.