All-solid-state rechargeable batteries embody the promise for high energy density, increased stability, and improved safety. However, their success is impeded by high resistance for mass and charge transfer at electrode-electrolyte interfaces. Li deficiency has been proposed as a major culprit for interfacial resistance, yet experimental evidence is elusive due to the challenges associated with noninvasively probing the Li distribution in solid electrolytes. In this Letter, three-dimensional 7Li magnetic resonance imaging (MRI) is employed to examine Li distribution homogeneity in solid electrolyte Li10GeP2S12 within symmetric Li/Li10GeP2S12/Li batteries. 7Li MRI and the derived histograms reveal Li depletion from the electrode-electrolyte interfaces and increased heterogeneity of Li distribution upon electrochemical cycling. Significant Li loss at interfaces is mitigated via facile modification with a poly(ethylene oxide)/bis(trifluoromethane)sulfonimide Li salt thin film. This study demonstrates a powerful tool for noninvasively monitoring the Li distribution at the interfaces and in the bulk of all-solid-state batteries as well as a convenient strategy for improving interfacial stability.