We report mid-infrared photocurrent spectra of graphene nanoribbon arrays on SiO2 dielectrics showing dual signatures of the substrate interaction. First, hybrid polaritonic modes of graphene plasmons and dielectric surface polar phonons produce a thermal photocurrent in graphene with spectral features that are tunable by gate voltage, nanoribbon width, and light polarization. Second, phonon polaritons associated with the substrate are excited, which indirectly heat up the graphene, leading to a graphene photocurrent with fixed spectral features. Models for other commonly used substrates show that the responsivity of graphene infrared photodetectors can be tailored to specific mid-IR frequency bands by the choice of the substrate.