Surface plasmon-enhanced vibrational spectroscopy has been demonstrated to be an important highly sensitive diagnostic technique, but its enhanced mechanism is yet to be explored. In this study, we couple femtosecond sum frequency generation vibrational spectroscopy (SFG-VS) with surface plasmon generated by the excitation of localized gold nanorods/nanoparticles and investigate the plasmonically enhanced factors (EFs) of SFG signals from poly(methyl methacrylate) films. Through monitoring the SFG intensity of carbonyl and ester methyl groups, we have established a correlation between EFs and the coupling of localized surface plasmon resonance with SFG and visible beams. It is found that the total enhanced factor is approximately proportional to the square of an enhanced factor of the SFG electromagnetic field and the fourth power of the enhanced factor of the visible electromagnetic field. The local field effect is roughly expressed to be the square of an enhanced factor of the visible electromagnetic field. This finding will help to guide the experimental design of plasmon-enhanced SFG to drastically improve the detection sensitivity and thus provide greater insight into the ultrafast dynamics near plasmonic surfaces.