The pH value of subcellular organelles in living cells is a significant parameter in the physiological activities of cells. Its abnormal fluctuations are commonly believed to be associated with cancers and other diseases. Herein, a series of surface-enhanced Raman scattering (SERS) nanosensors with high sensitivity and targeting function was prepared for the quantification and monitoring of pH values in mitochondria, nucleus, and lysosome. The nanosensors were composed of gold nanorods (AuNRs) functionalized with a pH-responsive molecule (4-mercaptopyridine, MPy) and peptides that could specifically deliver the AuNRs to the targeting subcellular organelles. The localization of our prepared nanoprobes in specific organelles was confirmed by super-high resolution fluorescence imaging and bio-transmission electron microscopy (TEM) methods. By the targeting ability, the pH values of the specific organelles can be determined by monitoring the vibrational spectral changes of MPy with different pH values. Compared to the cases of reported lysosome and cytoplasm SERS pH sensors, more accurate pH values of mitochondria and nucleus, which could be two additional intracellular tracers for subcellular microenvironments, were disclosed by this SERS approach, further improving the accuracy of discrimination of related diseases. Our sensitive SERS strategy can also be employed to explore crucial physiological and biological processes that are related to subcellular pH fluctuations.