Trans-to-cis photoisomerization efficiency of azobenzene dye is artificially modified from 0.09 to 0.38 when dye molecules are placed close to gold nanoparticle films with different structures. Transient fluorescence and surface enhanced Raman scattering measurement verify that the enhancement and reduction of photoisomerization efficiency come from the competition between enhanced local optical field from surface plasmon resonance and the accelerated nonradiative decay of excited dye molecules. The photoisomerization efficiency can be further modified by controlling the distance between azobenzene dye and gold films. Our finding can be applied to improve the performance of photoisomerization effect in photochemistry and photonics.