Prehistoric Japan underwent rapid transformations in the past 3000 years, first from foraging to wet rice farming and then to state formation. A long-standing hypothesis posits that mainland Japanese populations derive dual ancestry from indigenous Jomon hunter-gatherer-fishers and succeeding Yayoi farmers. However, the genomic impact of agricultural migration and subsequent sociocultural changes remains unclear. We report 12 ancient Japanese genomes from pre- and postfarming periods. Our analysis finds that the Jomon maintained a small effective population size of ~1000 over several millennia, with a deep divergence from continental populations dated to 20,000 to 15,000 years ago, a period that saw the insularization of Japan through rising sea levels. Rice cultivation was introduced by people with Northeast Asian ancestry. Unexpectedly, we identify a later influx of East Asian ancestry during the imperial Kofun period. These three ancestral components continue to characterize present-day populations, supporting a tripartite model of Japanese genomic origins.