We exploit the intrinsic structural instability of the Fe(70)Pd(30) magnetic shape memory alloy to obtain functional epitaxial films exhibiting a self-organized nanostructure. We demonstrate that coherent epitaxial straining by 54% is possible. The combination of thin film experiments and large-scale first-principles calculations enables us to establish a lattice relaxation mechanism, which is not expected for stable materials. We identify a low twin boundary energy compared to a high elastic energy as key prerequisite for the adaptive nanotwinning. Our approach is versatile as it allows to control both, nanostructure and intrinsic properties for ferromagnetic, ferroelastic, and ferroelectric materials.