Cultivated soybean (Glycine max) carrying the Rj2 allele restricts nodulation with specific Bradyrhizobium strains via host immunity, mediated by rhizobial type III secretory protein NopP and the host resistance protein Rj2. Here we found that the single isoleucine residue I490 in Rj2 is required for induction of symbiotic incompatibility. Furthermore, we investigated the geographical distribution of the Rj2-genotype soybean in a large set of germplasm by single nucleotide polymorphism (SNP) genotyping using a SNP marker for I490. By allelic comparison of 79 accessions in the Japanese soybean mini-core collection, we suggest substitution of a single amino acid residue (R490 to I490) in Rj2 induces symbiotic incompatibility with Bradyrhizobium diazoefficiens USDA 122. The importance of I490 was verified by complementation of rj2-soybean by the dominant allele encoding the Rj2 protein containing I490 residue. The Rj2 allele was also found in Glycine soja, the wild progenitor of G. max, and their single amino acid polymorphisms were associated with the Rj2-nodulation phenotype. By SNP genotyping against 1583 soybean accessions, we detected the Rj2-genotype in 5.4% of G. max and 7.7% of G. soja accessions. Distribution of the Rj2-genotype soybean plants was relatively concentrated in the temperate Asian region. These results provide important information about the mechanism of host genotype-specific symbiotic incompatibility mediated by host immunity and suggest that the Rj2 gene has been maintained by environmental conditions during the process of soybean domestication.