Rapid technology development and economic growth have brought attention to public health issues, such as food safety and environmental pollution, which creates an ever-increasing demand for fast and portable sensing technologies. Portable surface-enhanced Raman spectroscopy (SERS) capable of various analyte detection with low concentration in a convenient manner shows advantages in sensing technology including enhanced diagnostic precision, improved diagnostic efficiency, reduced diagnostic cost, and alleviation of patient pain, which emerges as a promising candidate for point-of-care testing (POCT). SERS detection technology based on different nanostructures made of noble metal-based nanomaterials can increase the sensitivity of Raman scattering by 6-8 orders of magnitude, making Raman based trace detection possible, and greatly promote the application scenarios of portable Raman spectrometers. In this perspective, we provide an overview of fundamental knowledge about the SERS mechanism including chemical and electromagnetic field enhancement mechanisms, the design and fabrication of SERS substrates based on materials, progress of using SERS for POCT in biochemical sensing and its clinical applications. Furthermore, we present the prospective of developing new nanomaterials with different functionalities for advanced SERS substrates, as well as the future advancement of biomedical sensing and clinical potential of SERS technology.