Infrared photoinduced force microscopy (IR-PiFM) is a scanning probe spectroscopic technique that maps sample morphology and chemical properties on the nanometer (nm)-scale. Fabricated samples with nm periodicity such as self-assembly of block copolymer films can be chemically characterized by IR-PiFM with relative ease. Despite the success of IR-PiFM, the origin of spectroscopic contrast remains unclear, preventing the scientific community from conducting quantitative measurements. Here we experimentally investigate the contrast mechanism of IR-PiFM for recording vibrational resonances. We show that the measured spectroscopic information of a sample is directly related to the energy lost in the oscillating cantilever, which is a direct consequence of a molecule excited at its vibrational optical resonance-coined as opto-mechanical damping. The quality factor of the cantilever and the local sample polarizability can be mathematically correlated, enabling quantitative analysis. The basic theory for dissipative tip-sample interactions is introduced to model the observed opto-mechanical damping.