Soft polymer films, such as polyelectrolyte multilayers (PEMs), are useful coatings in materials science. The properties of PEMs often rely on the degree of hydration, and therefore the study of these films in a hydrated state is critical to allow links to be drawn between their characteristics and performance in a particular application. In this work, we detail the development of a novel soft contact cell for studying hydrated PEMs (poly(sodium 4-styrenesulfonate)/poly(allylamine hydrochloride)) using FTIR microspectroscopy. FTIR spectroscopy can interrogate the nature of the polymer film and the hydration water contained therein. In addition to reporting spectra obtained for hydrated films confined at the solid-solid interface, we also report traditional ATR FTIR spectra of the multilayer. The spectra (microspectroscopy and ATR FTIR) reveal that the PEM film build-up proceeds as expected based on the layer-by-layer assembly methodology, with increasing signals from the polymer FTIR peaks with increasing bilayer number. In addition, the spectra obtained using the soft contact cell indicate that the PEM film hydration water has an environment/degree of hydrogen bonding that is affected by the chemistry of the multilayer polymers, based on differences in the spectra obtained for the hydration water within the film compared to that of bulk electrolyte.