Over-application of nitrogen fertilizer in fields has had a negative impact on both environment and human health. Domesticated rice varieties with high nitrogen use efficiency (NUE) reduce fertilizer for sustainable agriculture. Here, we perform genome-wide association analysis of a diverse rice population displaying extreme nitrogen-related phenotypes over three successive years in the field, and identify an elite haplotype of nitrate transporter OsNPF6.1HapB that enhances nitrate uptake and confers high NUE by increasing yield under low nitrogen supply. OsNPF6.1HapB differs in both the protein and promoter element with natural variations, which are differentially trans-activated by OsNAC42, a NUE-related transcription factor. The rare natural allele OsNPF6.1HapB, derived from variation in wild rice and selected for enhancing both NUE and yield, has been lost in 90.3% of rice varieties due to the increased application of fertilizer. Our discovery highlights this NAC42-NPF6.1 signaling cascade as a strategy for high NUE and yield breeding in rice.