We report on a highly sensitive measurement of the relative humidity of air, which utilizes a guided-mode resonance (GMR) of a multilayer dielectric structure (MDS) and the spectral interference of s- and p-polarized waves reflected from the MDS. We employ the MDS represented by four bilayers of TiO2/SiO2 with a termination layer of TiO2 and demonstrate that the GMR shows up as a shallow and asymmetric dip. The GMR enables us to measure the relative humidity (RH) of air with sensitivities of 0.031-0.114 nm/%RH. In addition, by employing a birefringent crystal of mica, which modifies the phase difference between the polarized waves, the GMR is transformed into the resonance with a sharp dip, and the measured sensitivity is enhanced to 0.120 nm/%RH at 81 %RH. We also determined the sensitivity to the refractive index and the figure of merit as high as 8000 nm/refractive index unit (RIU) and 702 RIU-1, respectively. The results demonstrate that the GMR based sensor employing the MDS and the spectral interference of polarized waves with their phase difference appropriately adjusted enables a highly sensitive, hysteresis-free humidity measurement, characterized by a high FOM. Humidity sensors employing dielectric multilayers thus represent an effective alternative to available sensors, with advantages such as better mechanical and chemical stability.