A fundamental goal in evolutionary biology is to understand the genetic architecture of adaptive traits. Using whole-genome data of 3955 of Darwin's finches on the Galápagos Island of Daphne Major, we identified six loci of large effect that explain 45% of the variation in the highly heritable beak size of Geospiza fortis, a key ecological trait. The major locus is a supergene comprising four genes. Abrupt changes in allele frequencies at the loci accompanied a strong change in beak size caused by natural selection during a drought. A gradual change in Geospiza scandens occurred across 30 years as a result of introgressive hybridization with G. fortis. This study shows how a few loci with large effect on a fitness-related trait contribute to the genetic potential for rapid adaptive radiation.