The sandy soil leaks water and fertilizer, and the ecological degradation is serious. The structural characteristics of soft rock and sandy soil are complementary, and the improvement of sandy soil by adding soft rock is of great significance to improve soil fertility, restore biodiversity, and maintain sustainable development of the Mu Us sandy land region. In this study, total organic carbon (TOC), particulate organic carbon (POC), dissolved organic carbon (DOC), easily oxidized organic carbon (ROC), microbial biomass carbon (SMBC), bacterial community structure and crop yield were examined using soft rock:sand volume ratios of 0:1 (CK), 1:5 (C1), 1:2 (C2) and 1:1 (C3). Our results indicated that, compared with the CK treatment, TOC (9.66-22.34%), POC (85.65-120.41%) and ROC (114.12-192.31%) noticeably increased in C1, C2 and C3 treatments; SMBC in treatment C3 increased by 42.77%. The three dominant bacteria in the soil (Proteobacteria, Actinobacteria and Chloroflexi), as well as Proteobacteria abundance, greatly declined in the treatments with the addition of soft rock. Pseudarthrobacter was the dominant Genus in all treatments, having an abundance between 11.83 and 19.33%. Bacterial diversity, richness and evenness indices all recorded an increase under the treatments. POC, TOC and SMBC recorded the most significant effects on the bacterial community structure. The largest increases in wheat and corn yields were recorded in the C2 treatment (16.05% and 16.30%), followed by the C1 treatment (8.28% and 8.20%, respectively). Our findings indicate that a soft rock:sand ratio between 1:5 and 1:2 recorded the most improvement in the sandy soil environment.