Addressing the spread of coronavirus disease 2019 (COVID-19) has highlighted the need for rapid, accurate, and low-cost diagnostic methods that detect specific antigens for SARS-CoV-2 infection. Tests for COVID-19 are based on reverse transcription PCR (RT-PCR), which requires laboratory services and is time-consuming. Here, by targeting the SARS-CoV-2 spike protein, we present a point-of-care SERS detection platform that specifically detects SARS-CoV-2 antigen in one step by captureing substrates and detection probes based on aptamer-specific recognition. Using the pseudovirus, without any pretreatment, the SARS-CoV-2 virus and its variants were detected by a handheld Raman spectrometer within 5 min. The limit of detection (LoD) for the pseudovirus was 124 TU μL-1 (18 fM spike protein), with a linear range of 250-10,000 TU μL-1. Moreover, this assay can specifically recognize the SARS-CoV-2 antigen without cross reacting with specific antigens of other coronaviruses or influenza A. Therefore, the platform has great potential for application in rapid point-of-care diagnostic assays for SARS-CoV-2.