We investigated the phylogenetic diversity and community structure of members of the halophilic Archaea (order Halobacteriales) in five distinct sediment habitats that experience various levels of salinity and salinity fluctuations (sediments from Great Salt Plains and Zodletone Spring in Oklahoma, mangrove tree sediments in Puerto Rico, sediment underneath salt heaps in a salt-processing plant, and sediments from the Great Salt Lake northern arm) using Halobacteriales-specific 16S rRNA gene primers. Extremely diverse Halobacteriales communities were encountered in all habitats, with 27 (Zodletone) to 37 (mangrove) different genera identified per sample, out of the currently described 38 Halobacteriales genera. With the exception of Zodletone Spring, where the prevalent geochemical conditions are extremely inhospitable to Halobacteriales survival, habitats with fluctuating salinity levels were more diverse than permanently saline habitats. Sequences affiliated with the recently described genera Halogranum, Halolamina, Haloplanus, Halosarcina, and Halorientalis, in addition to the genera Halorubrum, Haloferax, and Halobacterium, were among the most abundant and ubiquitous genera, suggesting a wide distribution of these poorly studied genera in saline sediments. The Halobacteriales sediment communities analyzed in this study were more diverse than and completely distinct from communities from typical hypersaline water bodies. Finally, sequences unaffiliated with currently described genera represented a small fraction of the total Halobacteriales communities, ranging between 2.5% (Zodletone) to 7.0% (mangrove and Great Salt Lake). However, these novel sequences were characterized by remarkably high levels of alpha and beta diversities, suggesting the presence of an enormous, yet-untapped supply of novel Halobacteriales genera within the rare biosphere of various saline ecosystems.