Graphene, one of the most robust two-dimensional (2D) materials, has shown amazing electrical, optical and thermal properties. The contact of graphene to other materials leads to Raman enhancement or fluorescence variation in some cases. Here, we present one flexible substrate structure of graphene/clustered-gold-nanoparticles (CGNs)/PDMS to demonstrate the Enhanced Surface Raman Spectroscopy. The fluorescence quenching on graphene is explained as ground-state charge transfer. In addition, graphene is found to possess the ability of fluorescence quenching or Raman enhancement when there are CGNs underneath. Moreover, graphene can be used as a passivation layer, protecting the CGNs underneath from chemical bonding with other materials. The experimental results also confirm that the total extra Raman enhancement is attributed to the extra electron shift to graphene and the extra ground state charge transfer introduced by the CGNs. This artificially engineered structures based on 2D materials could yield characteristics that go far beyond those conventional materials, and such materials should lead to higher sophistication of optical manipulation.