The surface-enhanced Raman spectroscopy (SERS) detection limit strongly depends on the molecular structure, which we demonstrate for a family of tert-butyl-substituted porphycenes. Even though the investigated species present very similar photophysical properties, the ratio between the SERS signal and fluorescence background depends on the number of bulky tert-butyl groups. Moreover, the probability of single molecule detection systematically drops with the number of the moieties attached to the pyrrole ring. As steric hindrance is the only significantly changing feature among the studied chromophores, we attribute the observed phenomena to the spatial structure. We also show that the sensitivity of the SERS technique can be improved by lowering the temperature. We managed to observe single-molecule spectra for derivatives for which this was unattainable at room temperature.