The Ebbinghaus illusion (EI) is an optical illusion of relative size perception that reflects the contextual integration ability in the visual modality. The current study investigated the genetic basis of two subtypes of EI, EI overestimation, and EI underestimation in humans, using quantitative genomic analyses. A total of 2825 Chinese adults were tested on their magnitudes of EI overestimation and underestimation using the method of adjustment, a standard psychophysical protocol. Heritability estimation based on common single nucleotide polymorphisms (SNPs) revealed a moderate heritability (34.3%) of EI overestimation but a nonsignificant heritability of EI underestimation. A meta-analysis of two phases (phase 1: n = 1986, phase 2: n = 839) of genome-wide association study (GWAS) discovered 1969 and 58 SNPs reaching genome-wide significance for EI overestimation and EI underestimation, respectively. Among these SNPs, 55 linkage-disequilibrium-independent SNPs were associated with EI overestimation in phase 1 with genome-wide significance and their associations could be confirmed in phase 2 cohort. Gene-based analyses found seven genes to be associated with EI overestimation at the genome-wide level, two from meta-analysis, and five from classical two-stage analysis. Overall, this study provided consistent evidence for a substantial genetic basis of the Ebbinghaus illusion.