Kernel row number (KRN) is an important component of yield during the domestication and improvement of maize and controlled by quantitative trait loci (QTL). Here, we fine-mapped a major KRN QTL, KRN4, which can enhance grain productivity by increasing KRN per ear. We found that a ~3-Kb intergenic region about 60 Kb downstream from the SBP-box gene Unbranched3 (UB3) was responsible for quantitative variation in KRN by regulating the level of UB3 expression. Within the 3-Kb region, the 1.2-Kb Presence-Absence variant was found to be strongly associated with quantitative variation in KRN in diverse maize inbred lines, and our results suggest that this 1.2-Kb transposon-containing insertion is likely responsible for increased KRN. A previously identified A/G SNP (S35, also known as Ser220Asn) in UB3 was also found to be significantly associated with KRN in our association-mapping panel. Although no visible genetic effect of S35 alone could be detected in our linkage mapping population, it was found to genetically interact with the 1.2-Kb PAV to modulate KRN. The KRN4 was under strong selection during maize domestication and the favorable allele for the 1.2-Kb PAV and S35 has been significantly enriched in modern maize improvement process. The favorable haplotype (Hap1) of 1.2-Kb-PAV-S35 was selected during temperate maize improvement, but is still rare in tropical and subtropical maize germplasm. The dissection of the KRN4 locus improves our understanding of the genetic basis of quantitative variation in complex traits in maize.