Learning microscopic properties of a material from its macroscopic measurables is a grand and challenging goal in physical science. Conventional wisdom is to first identify material structures exploiting characterization tools, such as spectroscopy, and then to infer properties of interest, often with assistance of theory and simulations. This indirect approach has limitations due to the accumulation of errors from retrieving structures from spectral signals and the lack of quantitative structure-property relationship. A new pathway directly from spectral signals to microscopic properties is highly desirable, as it would offer valuable guidance toward materials evaluation and design via spectroscopic measurements. Herein, we exploit machine-learned vibrational spectroscopy to establish quantitative spectrum-property relationships. Key interaction properties of substrate-adsorbate systems, including adsorption energy and charge transfer, are quantitatively determined directly from Infrared and Raman spectroscopic signals of the adsorbates. The machine-learned spectrum-property relationships are presented as mathematical formulas, which are physically interpretable and therefore transferrable to a series of metal/alloy surfaces. The demonstrated ability of quantitative determination of hard-to-measure microscopic properties using machine-learned spectroscopy will significantly broaden the applicability of conventional spectroscopic techniques for materials design and high throughput screening under operando conditions.