Sulfite-reducing and sulfate-reducing microorganisms (SRM) play important roles in anoxic environments, linking the sulfur and carbon cycles. With climate warming, the distribution of anoxic habitats conductive to dissimilatory SRM is expanding. Consequently, we hypothesize that novel SRM are likely to emerge from the rare biosphere triggered by environmental changes. Using the dsrB gene as a molecular marker of sulfite-reducers and sulfate-reducers, we analyzed the diversity, community composition, and abundance of SRM in 200 samples representing 14 different ecosystems, including marine and freshwater environments, oil reservoirs, and engineered infrastructure. Up to 167,397 species-level OTUs affiliated with 47 different families were identified. Up to 96% of these can be considered as "rare biosphere SRM". One third of the dsrB genes identified belonged to uncharacterized lineages. The dsrB sequences exhibited a strong pattern of selection in different ecosystems. These results expand our knowledge of the biodiversity and distribution of SRM, with implications for carbon and sulfur cycling in anoxic ecosystems.