We derive general coupled-mode equations describing the nonlinear interaction of electromagnetic modes in periodic media with loss and gain. Our approach is rigorously based on the Lorentz reciprocity theorem, and it can be applied to a broad range of metal-dielectric photonic structures, including plasmonic waveguides and metamaterials. We verify that our general results agree with the previous analysis of particular cases, and predict novel effects on self- and cross-phase modulation in multilayer nonlinear fishnet metamaterials.