Photoelectrochemical devices could play a crucial role toward fuel production in a circular economy. Yet, light absorption suffers losses from thermalization and the inability to use low-energy photons. Here, we demonstrate that photoelectrochemical reactors can utilize this waste heat by integrating thermoelectric modules, which provide additional voltage under concentrated light irradiation. While most single semiconductors require external bias, we already accomplish unassisted water splitting under 2 sun irradiation by wiring a BiVO4 photoanode to a thermoelectric element, whereas the photocurrent of a perovskite-BiVO4 tandem system is enhanced 1.7-fold at 5 sun. This strategy is particularly suitable for photoanodes with more positive onset potentials like hematite, with thermoelectric-perovskite-Fe2O3 systems achieving a 29.7× overall photocurrent increase at 5 sun over conventional perovskite-Fe2O3 devices without light concentration. This thermal management approach provides a universal strategy to facilitate widespread solar fuel production, as light concentration increases output, reduces the reactor size and cost, and may enhance catalysis.