Plasmonic nanoparticle assemblies have been exhibiting unique collective properties absent in their individual counterparts. However, it is an important challenge to manipulate those properties due to the difficulty in controlling the arrangement and distance between plasmonic nanoparticles. Herein, we propose an alternative strategy for manipulating the distance between gold nanoparticles on the plasmonic vesicles to afford tunable collective properties by changing the temperature. To reach this goal, a thermally responsive vesicle is self-assembled from an azobenzene-terminated homopolymer, poly(2-(2-ethoxyethoxy)ethyl acrylate) (Azo-PEEA). Gold nanoparticles are then decorated on its membrane to afford plasmonic vesicles, which can be grouped and fused into larger plasmonic vesicles when heated. Consequently, the gold nanoparticles come closer, creating local hot spots in the gap between adjacent gold nanoparticles, leading to the red shift of local surface plasmon resonance (LSPR) peaks and better surface-enhanced Raman scattering (SERS). Besides, the structure and the collective optical properties of the plasmonic vesicles can be reserved under various conditions, e.g., different pH values, high salt concentration and relatively high temperature once they are heated up to 35 °C.