Graphene field effect transistors (FETs) are extremely sensitive to gas exposure. Charge transfer doping of graphene FETs by atmospheric gas is ubiquitous but not yet understood. We have used graphene FETs to probe minute changes in electrochemical potential during high-purity gas exposure experiments. Our study shows quantitatively that electrochemistry involving adsorbed water, graphene, and the substrate is responsible for doping. We not only identify the water/oxygen redox couple as the underlying mechanism but also capture the kinetics of this reaction. The graphene FET is highlighted here as an extremely sensitive potentiometer for probing electrochemical reactions at interfaces, arising from the unique density of states of graphene. This work establishes a fundamental basis on which new electrochemical nanoprobes and gas sensors can be developed with graphene.