Despite existing experimental and computational tools to assess the risk, the non-specific chemical modification of protein thiol groups remains a significant source of false-positive hits, particularly in academic drug discovery. Herein, we describe the application of a simple NMR method in a systematic study on the reactivity of 5-benzylidenebarbiturates, 5-benzylidenerhodanines, and their related oxo-heterocycles, which have been associated with numerous biological activities and have recently gained a reputation as unselective promiscuous binders. Using this method, we confirmed the reactivity of 5-benzylidenebarbiturates, which are known to easily form Michael adducts with nucleophiles. In contrast, 5-benzylidene five-membered oxo-heterocycles revealed almost insignificant reactivity. We can conclude that the distinct binding profile of the most controversial compounds, 5-benzylidenerhodanines, is not necessarily related to their unspecific Michael acceptor reactivity.