Intratumoral heterogeneity is a substantial cause of drug resistance development during chemotherapy or other drug treatments for cancer. Therefore, monitoring and measuring cell exposure and response to drugs at the single-cell level are crucial. Previous research suggested that the single-cell growth rate can be used to investigate drug-cell interactions. However, currently established methods for quantifying single-cell growth are limited to isolated or monolayer cells. Here, we introduce a technique that accurately measures both 2D and 3D cell growth rates using label-free ratiometric stimulated Raman scattering (SRS) microscopy. We use deuterated amino acids, leucine, isoleucine, and valine, as tracers and measure the C-D SRS signal from deuterium-labeled proteins and the C-H SRS signal from unlabeled proteins simultaneously to determine the cell growth rate at the single-cell level. The technique offers single-cell level drug sensitivity measurement with a shorter turnaround time (within 12 h) than most traditional assays. The submicrometer resolution of the imaging technique allows us to examine the effects of chemotherapeutic drugs, including kinase inhibitors, mitotic inhibitors, and topoisomerase II inhibitors, on both the cell growth rate and morphology. The capability of quantifying 3D cell growth rates provides insight into a deeper understanding of the cell-drug interaction in the actual tumor environment.