Cold water oligomers (H2O)n and (D2O)n with n = 2-5 are assigned in spontaneous Raman scattering spectra of seeded rare gas expansions for the first time. Comparison with infrared spectra provides direct experimental insights into the hydrogen bond-mediated excitonic OH oscillator coupling, which is responsible for ultrafast energy transfer between water molecules, usually suppressed by isotopic dilution in femtosecond experiments for the condensed phase. The experimental coupling constants are compared to those in state-of-the-art full-dimensional water potential energy hypersurfaces, leaving room for improvement in the description of the coupled dynamics in water. Evidence for intensified Fermi resonance between OH stretching and OH bending motion beyond water trimers is collected.