The plasmonic amplification of nonlinear vibrational sum frequency spectroscopy (SFG) at the surfaces of gold nanoparticles is systematically investigated by tuning the incident visible wavelength. The SFG spectra of dodecanethiol-coated gold nanoparticles chemically deposited on silicon are recorded for 20 visible wavelengths. The vibrational intensities of thiol methyl stretches extracted from the experimental measurements vary with the visible color of the SFG process and show amplification by coupling to plasmon excitation. Because the enhancement is maximal in the orange-red region rather than in the green, as expected from the dipolar model for surface plasmon resonances, it is attributed mostly to hotspots created in particle multimers, in spite of their low surface densities. A simple model accounting for the longitudinal surface plasmons of multimers allows the recovery of the experimental spectral dispersion.