The quantitative trait loci (QTL) for porcine ear size was previously reported to mainly focus on SSC5 and SSC7. Recently, a missense mutation, G32E, in PPARD in the QTL interval on SSC7 was identified as the causative mutation for ear size. However, on account of the large interval of QTL, the responsible gene on SSC5 has not been identified. In this study, an intercross population was constructed from the large-eared Minzhu, an indigenous Chinese pig breed, and the Western commercial Large White pig to examine the genetic basis of ear size diversity. A GWAS was performed to detect SNPs significantly associated with ear size. Thirty-five significant SNPs defined a 10.78-Mb (30.14-40.92 Mb) region on SSC5. Further, combining linkage disequilibrium and haplotype sharing analysis, a reduced region of 3.07-Mb was obtained. Finally, by using a selective sweep analysis, a critical region of about 450-kb interval containing two annotated genes LEMD3 and WIF1 was refined in this work. Functional analysis indicated that both represent biological candidates for porcine ear size, with potential application in breeding programs. The two genes could also be used as novel references for further study of the mechanism underlying human microtia.