In this report, we have developed a novel surface-enhanced Raman scattering (SERS) aptasensor for ricin B toxin recognition based on Ag nanoparticles labeled with 4,4'-bipyridyl (Bpy, Raman reporter) and ricin B aptamer. The hybrid silicon substrate was first prepared via surface-mediated RAFT polymerization of N-acryoyl-l-valine in the presence of 2-(butylthio-carbonothioylthio)-2-methylpropionic acid-modified silicon wafer as a RAFT agent and then biofunctionalized with ricin B aptamer. In this novel system, the ricin B aptamer functionalized silicon substrate was used as a scavenger for target ricin B molecules. After ricin B molecules were separated from the matrix, the sandwich assay procedure was applied using Ag nanoparticles labeled with Bpy and ricin B aptamer which act as SERS probes. Meanwhile, to enhance the SERS signal, silver deposition on the sandwich complex was also performed. The correlation between the ricin B concentration and SERS signal was found to be linear within the range of 1.0 fM to 50 pM. The limit of detection for the SERS aptasensor was determined as 0.32 fM. Furthermore, the SERS aptasensor was also evaluated for detecting ricin B in artificially contaminated orange juice, milk, blood and urine. Finally, this method has the potential to be used for the detection of other protein toxins in a complex matrix if a specific aptamer for that protein toxin can be designed.