Breast cancer is the most commonly diagnosed cancer type worldwide. Overexpression of human epidermal growth factor receptor 2 (HER2) is an important subtype of breast cancer and results in an increased risk of recurrence and metastasis in patients. At present, immunohistochemistry (IHC) is used to detect the expression of HER2 in breast cancer tissues as the golden standard. However, IHC has some shortcomings, such as large subjective impact, long time consumption, expensive reagents, etc. In this paper, a combined morphological and spectroscopic diagnostic method based on label-free surface-enhanced Raman scattering (SERS) for HER2 expression in breast cancer is proposed. It can not only quantitively detect HER2 expression in breast cancer tissues by spectroscopic measurements but also give morphological images reflecting the distribution of HER2 in tissues. The results show that the consistency between this method and IHC is 95% and achieves the annotation of tumor regions on tissue sections. This method is time-consuming, quantifiable, intuitive, scalable, and easy to understand. Combined with deep learning approaches, it is expected to promote the development of clinical detection and diagnosis technology for breast cancer and other cancers.