This paper reviews current experimental and theoretical progress in the study of dipolar quantum gases of ground and meta-stable atoms with a large magnetic moment. We emphasize the anisotropic nature of Feshbach resonances due to coupling to fast-rotating resonant molecular states in ultracold s-wave collisions between magnetic atoms in external magnetic fields. The dramatic differences in the distribution of resonances of magnetic (7)S3 chromium and magnetic lanthanide atoms with a submerged 4f shell and non-zero electron angular momentum is analyzed. We focus on dysprosium and erbium as important experimental advances have been recently made to cool and create quantum-degenerate gases for these atoms. Finally, we describe progress in locating resonances in collisions of meta-stable magnetic atoms in electronic P-states with ground-state atoms, where an interplay between collisional anisotropies and spin-orbit coupling exists.