The successful implementation of heterosis in rice has significantly enhanced rice productivity, but the genetic basis of heterosis in rice remains unclear. To understand the genetic basis of heterosis in rice, main-effect and epistatic quantitative trait loci (QTLs) associated with heterosis for grain yield-related traits in the four related rice mapping populations derived from Xiushui09 (XS09) (japonica) and IR2061 (indica), were dissected using single nucleotide polymorphism bin maps and replicated phenotyping experiments under two locations. Most mid-parent heterosis of testcross F1s (TCF1s) of XS09 background introgression lines (XSILs) with Peiai64S were significantly higher than those of TCF1s of recombinant inbred lines (RILs) with PA64S at two locations, suggesting that the effects of heterosis was influenced by the proportion of introgression of IR2061's genome into XS09 background. A total of 81 main-effect QTLs (M-QTLs) and 41 epistatic QTLs were identified for the phenotypic variations of four traits of RILs and XSILs, TCF1s and absolute mid-parent heterosis in two locations. Furthermore, overdominance and underdominance were detected to play predominant effects on most traits in this study, suggesting overdominance and underdominance as well as epistasis are the main genetic bases of heterosis in rice. Some M-QTLs exhibiting positive overdominance effects such as qPN1.2, qPN1.5 and qPN4.3 for increased panicle number per plant, qGYP9 and qGYP12.1 for increased grain yield per plant, and qTGW3.4 and qTGW8.2 for enhanced 1000-grain weight would be highly valuable for breeding to enhance grain yield of hybrid rice by marker-assisted selection.