In this Letter, we show that the electromagnetic duality symmetry, broken in the microscopic Maxwell's equations by the presence of charges, can be restored for the macroscopic Maxwell's equations. The restoration of this symmetry is shown to be independent of the geometry of the problem. These results provide a tool for the study of light-matter interactions within the framework of symmetries and conservation laws. We illustrate its use by determining the helicity content of the natural modes of structures possessing spatial inversion symmetries and by elucidating the root causes for some surprising effects in the scattering off magnetic spheres.