The trace detection of toxic compounds in complex matrices is a major concern, in particular when it comes to mycotoxins in food. We developed a highly sensitive and specific SERS sensor for the detection of ochratoxin A using a simple rough gold film as a substrate. When adding the analyte, we observed spectral variations related to the interaction of the analyte with the specific aptamer used as a bioreceptor. Using a partial least squares regression method, our sensor is able to detect concentrations down to the picomolar range, which is much lower than the minimum legal concentration allowed in food products. Moreover, we demonstrate the accurate detection of the analyte in a wide concentration range from the picomolar up to the micromolar level. The detection was validated with negative detection tests using deoxynivalenol and bovine serum albumin.