The AMnO2 delafossites (A = Na, Cu) are model frustrated antiferromagnets, with triangular layers of Mn3+ spins. At low temperatures (TN = 65 K), a C2/m → P1̅ transition is found in CuMnO2, which breaks frustration and establishes magnetic order. In contrast to this clean transition, A = Na only shows short-range distortions at TN. Here, we report a systematic crystallographic, spectroscopic, and theoretical investigation of CuMnO2. We show that, even in stoichiometric samples, nonzero anisotropic Cu displacements coexist with magnetic order. Using X-ray/neutron diffraction and Raman scattering, we show that high pressures act to decouple these degrees of freedom. This manifests as an isostuctural phase transition at ∼10 GPa, with a reversible collapse of the c-axis. This is shown to be the high-pressure analogue of the c-axis negative thermal expansion seen at ambient pressure. Density functional theory (DFT) simulations confirm that dynamical instabilities of the Cu+ cations and edge-shared MnO6 layers are intertwined at ambient pressure. However, high pressure selectively activates the former, before an eventual predicted reemergence of magnetism at the highest pressures. Our results show that the lattice dynamics and local structure of CuMnO2 are quantitatively different from nonmagnetic Cu delafossites and raise questions about the role of intrinsic inhomogeneity in frustrated antiferromagnets.