Amphetamine (AMP), as a psychiatric drug acting on the central nervous system, and has become one of the most common drugs of abuse in the illegal market at present, which adversely affects social public safety. We developed a SERS magnetic immunoassay with high sensitivity, specificity, and rapid and quantitative detection of AMP. We synthesized a high SERS intensity substrate (Au-XP013@Ag) using the "hot spot" effect and combined it with antibodies to form SERS immunotags (Au-XP013@Ag-AMP-mAb). Subsequently, the carboxyl magnetic beads were linked to label antigens as functional magnetic beads (carboxyl magnetic beads-AMP-BSA). Using the principle of competitive immunoassay, the Raman response value of the immune complex formed with SERS tags and functional magnetic beads was detected to realize the quantitative detection of AMP. The detection limit of this method for AMP was 2.28 ng mL-1. More importantly, a portable Raman instrument was used in this study, which can meet the requirements of point-of-care testing (POCT). Therefore, this SERS-based magnetic immunoassay can provide a favorable scientific basis for the control of drug abuse, monitoring by law enforcement agencies, and determination of drug users.