The responsivity of graphene photodetectors depends critically on the elevated temperature of the electronic subsystem upon photoexcitation. We investigate the role of the substrate in providing cooling pathways for photoexcited carriers under ambient conditions by partially suspending few-layer graphene over a trench. Through photocurrent microscopy, we observe p-n junctions near the supported/suspended interfaces that produce photothermoelectric currents. Most importantly, we find the photocurrent in suspended p-n junctions to be an order of magnitude larger than in supported structures. This enhancement is attributed to the elimination of a dominant electronic cooling channel via the surface phonons of the polar substrate. Our work documents this mechanism of energy exchange between graphene and its environment, and it points to the importance of dielectric engineering for future improved graphene photodetectors.