Uncooled infrared detectors have enabled the rapid growth of thermal imaging applications. These detectors are predominantly bolometers, reading out a pixel's temperature change due to infrared radiation as a resistance change. Another uncooled sensing method is to transduce the infrared radiation into the frequency shift of a mechanical resonator. We present here highly sensitive resonant infrared sensors, based on thermo-responsive shape memory polymers. By exploiting the phase-change polymer as transduction mechanism, our approach provides 2 orders of magnitude improvement of the temperature coefficient of frequency. Noise equivalent temperature difference of 22 mK in vacuum and 112 mK in air are obtained using f/2 optics. The noise equivalent temperature difference is further improved to 6 mK in vacuum by using high-Q silicon nitride membranes as substrates for the shape memory polymers. This high performance in air eliminates the need for vacuum packaging, paving a path towards flexible non-hermetically sealed infrared sensors.