Rapid testing of cement raw meal plays a crucial role in the cement production process, so there is an urgent need for a fast and accurate testing method. In this paper, a method based on the Savitzky-Golay (SG) smoothing and sample set partitioning based on joint x-y distance (SPXY) spectral data pre-processing is proposed to improve the accuracy of the laser-induced breakdown spectroscopy (LIBS) technique for quantitative analysis of cement raw meal components. Firstly, the spectral data is denoised by SG smoothing, which effectively reduces the noise and baseline variations in the spectra. Then, the denoised data is divided into sample sets by combining the SPXY sample division method, which improves the efficiency of data analysis. Finally, the delineated data set is modeled for quantitative analysis by a back-propagation (BP) neural network. Compared to the modeling effect of the four oxide contents of CaO, S i O 2, A l 2 O 3, and F e 2 O 3 in the Hold-Out method, the correlation coefficient (R) was improved by 26%, 10%, 17%, and 4%, respectively. The root mean square error (RMSE) was reduced by 47%, 33%, 43%, and 21%, respectively. The mean absolute percentage error (MAPE) was reduced by 63%, 60%, 36%, and 51%, respectively. The results show that there is a significant improvement in the model effect, which can effectively improve the accuracy of quantitative analysis of cement raw meal composition by LIBS. This is of great significance for the real-time detection of cement raw meal composition analysis.