Voltage losses are one of the main obstacles for further improvement in the power conversion efficiency of organic solar cells. In this work, we investigate the effect of thermal stress on voltage losses in various material systems by multiple spectroscopic measurements on both devices and thin films. The energetics of nonfullerene small molecules are more readily altered under thermal stress compared to all-polymer and fullerene-based systems, thereby strongly affecting open-circuit voltage. These energetics variations correlate with the glass transition of respective materials. While nonfullerene small molecular acceptor systems exhibit both dynamic and static disorders which can be restrained in annealed films, all-polymeric systems exhibit dominated static disorders, which are also stable against thermal stress. The much higher voltage losses in fullerene-based systems compared to the other two counterparts are mainly due to the losses from device band gap to charge transfer states and the high nonradiative recombination.