Over the past 30 years, human disturbance and habitat fragmentation have severely endangered the survival of common wild rice (Oryza rufipogon Griff.) in China. A better understanding of the genetic structure of O. rufipogon populations will therefore be useful for the development of conservation strategies. We examined the diversity and genetic structure of natural O. rufipogon populations at the national, provincial, and local levels using simple sequence repeat (SSR) markers. Twenty representative populations from sites across China showed high levels of genetic variability, and approximately 44% of the total genetic variation was among populations. At the local level, we studied fourteen populations in Guangxi Province and four populations in Jiangxi Province. Populations from similar ecosystems showed less genetic differentiation, and local environmental conditions rather than geographic distance appeared to have influenced gene flow during population genetic evolution. We identified a triangular area, including northern Hainan, southern Guangdong, and southwestern Guangxi, as the genetic diversity center of O. rufipogon in China, and we proposed that this area should be given priority during the development of ex situ and in situ conservation strategies. Populations from less common ecosystem types should also be given priority for in situ conservation.