In this paper, we ask whether rapidly displacing coal electricity generation with underutilized, existing natural gas capacity has net societal benefits or net costs when considering climate change, economics, and air pollution. We use a power plant dispatch model to quantify the effects of dispatching because of a tax on carbon or because of a tax on carbon, methane leakage, and air pollution. We explicitly model exhaust stack CO2 emissions, production costs, health damages caused by criteria air pollutants, and methane leakage from the natural gas infrastructure. We show that (1) the optimal coal-to-gas redispatch displaces 62-77% of coal energy, leaving some coal online, (2) the health benefits of redispatch are larger in magnitude than the climate benefits, (3) reducing methane leakage rates from 2.3 to 2.0% increases the net climate benefits of redispatch by $1.1B-$1.4B, (4) although internalizing methane leakage, climate damages, and health damages in the power plant dispatch maximizes the net benefits of redispatch, 75-87% of these benefits can be achieved using a carbon tax mechanism alone, and (5) when choosing an optimal carbon tax, focusing on climate at the exclusion of health-and vice-versa-provides less net benefit than looking at both issues jointly.