Thermo-optical tuning of the refractive index is one of the pivotal operations performed in integrated silicon photonic circuits for thermal stabilization, compensation of fabrication tolerances, and implementation of photonic operations. Currently, heaters based on metal wires provide the temperature control in the silicon waveguide. The strong interaction of metal and light, however, necessitates a certain gap between the heater and the photonic structure to avoid significant transmission loss. Here we present a graphene heater that overcomes this constraint and enables an energy efficient tuning of the refractive index. We achieve a tuning power as low as 22 mW per free spectral range and fast response time of 3 µs, outperforming metal based waveguide heaters. Simulations support the experimental results and suggest that for graphene heaters the spacing to the silicon can be further reduced yielding the best possible energy efficiency and operation speed.