Water is one of the principal constituents by mass of living plant cell walls. However, its role and interactions with secondary cell wall polysaccharides and the impact of dehydration and subsequent rehydration on the molecular architecture are still to be elucidated. This work combines multidimensional solid-state 13C magic-angle-spinning (MAS) nuclear magnetic resonance (NMR) with molecular dynamics modeling to decipher the role of water in the molecular architecture of softwood secondary cell walls. The proximities between all main polymers, their molecular conformations, and interaction energies are compared in never-dried, oven-dried, and rehydrated states. Water is shown to play a critical role at the hemicellulose-cellulose interface. After significant molecular shrinkage caused by dehydration, the original molecular conformation is not fully recovered after rehydration. The changes include xylan becoming more closely and irreversibly associated with cellulose and some mannan becoming more mobile and changing conformation. These irreversible nanostructural changes provide a basis for explaining and improving the properties of wood-based materials.