The assessment of the muscle invasion of bladder cancer typically plays a crucial role in therapeutic decision-making and has significant impacts on the recurrence rate and survival rate. Although histopathology is sufficiently accurate and usually served as the gold standard for bladder cancer diagnosis, it is invasive, time-consuming, and requires intensive sample preparation by a well-trained pathologist to achieve an optimal diagnosis. Therefore, a fast and noninvasive method to accurately identify non-muscle-invasive bladder cancer (NMIBC) and muscle-invasive bladder cancer (MIBC) is in demand. In this study, the SERS technique combined with the PLS-LDA method based on a small amount of blood serum samples is employed to distinguish healthy volunteers, NMIBC, and MIBC patients. According to the results, the overall diagnostic accuracy is 93.3%. The diagnostic accuracies are 97.8% and 93.2% for healthy versus bladder cancer groups and NMIBC versus MIBC groups, respectively. Therefore, the proposed method has demonstrated excellent performance on accurately identifying muscle invasion of bladder cancer, which can assist timely diagnosis and proper treatment for bladder cancer patients.