Surface-enhanced Raman scattering (SERS) spectroscopy analysis has long been the central task of nanoscience and nanotechnology to realize the ultrasensitive recognition/quantitation applications. Recently, the blooming of artificial intelligence algorithms provides an edge tool to revolutionize the spectroscopy analysis, especially for multiple substances analysis and large-scale data handling. In this study, a single-model multi-tasks deep learning network is proposed to simultaneously achieve the qualitative recognition and quantitative analysis of SERS spectroscopy. The SERS spectra of two kinds of hypoglycemic drugs (PHE, ROS) and the corresponding mixtures are collected, respectively, with the concentration grade from 10-4 M to 10-8 M. Based on the SERS spectroscopy dataset, the loss functions and hyperparameters of the multi-tasks classifications model are optimized, and the recognition accuracies are tested by simulation experiments. It is demonstrated that the accuracy rates of qualitative and quantitative analysis can reach up to 99.0% and 98.4%, respectively. Moreover, the practical feasibility of this multi-tasks model is demonstrated by using it to achieve qualitative and quantitative analysis of PHE and ROS in complex serum matrix. Overall, this single-model multi-tasks deep learning network shows significant potential for the recognition and quantitation of SERS spectroscopy, which provides the algorithmic and experimental basis for large-scale and multiple substances SERS spectra analysis.