Phosphate is an essential component of cell functions, and the specific transport of phosphorus into a cell is mediated by phosphate-binding protein (PBP). The mechanism of PBP-phosphate recognition remains controversial: on the basis of similar binding affinities at acidic and basic pHs, it is believed that the hydrogen network in the binding site is flexible to adapt to different protonation states of phosphates. However, only hydrogen (1H) phosphate was observed in the sub-angstrom X-ray structures. To address this inconsistency, we performed molecular dynamics simulations using the AMOEBA polarizable force field. Structural and free energy data from simulations suggested that 1H phosphate was the preferred bound form at both pHs. The binding of dihydrogen (2H) phosphate disrupted the hydrogen-bond network in the PBP pocket, and the computed affinity was much weaker than that of 1H phosphate. Furthermore, we showed that the discrepancy in the studies described above is resolved if the interaction between phosphate and the buffer agent is taken into account. The calculated apparent binding affinities are in excellent agreement with experimental measurements. Our results suggest the high specificity of PBP for 1H phosphate and highlight the importance of the buffer solution for the binding of highly charged ligands.