One of the main challenges for highly sensitive surface-enhanced Raman scattering (SERS) detection is the noise interference of fluorescence signals arising from the analyte molecules. Here we used three types of gold nanostars (GNSs) SERS probes treated by different surface modification methods to reveal the simultaneously existed Raman scattering enhancement and inhibiting fluorescence behaviors during the SERS detection process. As the distance between the metal nanostructures and the analyte molecules can be well controlled by these three surface modification methods, we demonstrated that the fluorescence signals can be either quenched or enhanced during the detection. We found that fluorescence quenching will occur when analyte molecules are closely contacted to the surface of GNSs, leading to a ~100 fold enhancement of the SERS sensitivity. An optimized Raman signal detection limit, as low as the level of 10-11 M, were achieved when Rhodamine 6 G were used as the analyte. The presented fluorescence-free GNSs SERS substrates with plentiful hot spots and controllable surface plasmon resonance wavelengths, fabricated using a cost-effective self-assembling method, can be very competitive candidates for high-sensitive SERS applications.