The instability of currently used electrolyte solutions and of the carbon support during charge-discharge in non-aqueous lithium-oxygen cells can lead to discharge products other than the desired Li2O2, such as Li2CO3, which is believed to reduce cycle-life. Similarly, discharge in an O2 atmosphere which contains H2O and CO2 impurities would lead to LiOH and Li2CO3 discharge products. In this work we therefore investigate the rechargeability of model cathodes pre-filled with four possible Li-air cell discharge products, namely Li2O2, Li2CO3, LiOH, and Li2O. Using Online Electrochemical Mass Spectrometry (OEMS), we determined the charge voltages and the gases evolved upon charge of pre-filled electrodes, thus determining the reversibility of the formation/electrooxidation reactions. We show that Li2O2 is the only reversible discharge product in ether-based electrolyte solutions, and that the formation of Li2CO3, LiOH, or Li2O is either irreversible and/or reacts with the electrolyte solution or the carbon during its oxidation.