Histidine kinases are key regulators in the bacterial two-component systems that mediate the cellular response to environmental changes. The vast majority of the sensor histidine kinases belong to the bifunctional HisKA family, displaying both kinase and phosphatase activities toward their substrates. The molecular mechanisms regulating the opposing activities of these enzymes are not well understood. Through a combined NMR and crystallographic study on the histidine kinase HK853 and its response regulator RR468 from Thermotoga maritima, here we report a pH-mediated conformational switch of HK853 that shuts off its phosphatase activity under acidic conditions. Such a pH-sensing mechanism is further demonstrated in the EnvZ-OmpR two-component system from Salmonella enterica in vitro and in vivo, which directly contributes to the bacterial infectivity. Our finding reveals a broadly conserved mechanism that regulates the phosphatase activity of the largest family of bifunctional histidine kinases in response to the change of environmental pH.