The question of how genotypic and ecological units arise and spread in natural microbial populations remains controversial in the field of evolutionary biology. Here, we investigated the early stages of ecological and genetic differentiation in a highly clonal sympatric Sinorhizobium meliloti population. Whole-genome sequencing revealed that a large DNA region of the symbiotic plasmid pSymB was replaced in some isolates with a similar synteny block carrying densely clustered SNPs and displaying gene acquisition and loss. Two different versions of this genomic island of differentiation (GID) generated by multiple genetic exchanges over time appear to have arisen recently, through recombination in a particular clade within this population. In addition, these isolates display resistance to phages from the same geographic region, probably due to the modification of surface components by the acquired genes. Our results suggest that an underlying process of early ecological and genetic differentiation in S. meliloti is primarily triggered by acquisition of genes that confer resistance to soil phages within particular large genomic DNA regions prone to recombination.