The Bph1 gene was the first reported brown planthopper (BPH, Nilaparvata lugens) resistance gene in Mudgo rice and was widely used as a commercial cultivar for controlling BPH infestations. However, rapid adaptations of BPH on the Mudgo rice resulted in its resistance breakdown and the emergence of virulent BPH populations. Thus, specific BPH populations and rice varieties can serve as good model systems for studying the roles of different bio-compounds and proteins in the insect-plant interactions. Although our understandings have been improved on the complexity of BPH and rice interactions, the underlying molecular mechanisms remain largely unknown. Here we analyzed the feeding performances and the transcriptomic responses of two BPH populations (Mugdo-BPH and TN1-BPH) during compatible (Mudog-BPH feeding on Mudgo rice) and incompatible (TN1-BPH feeding on Mudgo rice) interactions. The electrical penetration graph (EPG) results indicated that the BPH feeding and performances during the incompatible interaction are significantly affected in terms of decreased honeydew, loss of weight, decreased phloem sap ingestion (N4 waveform), but increased non-penetration (NP waveform) phase. Abundance of glucose and trehalose was reduced in BPH during the incompatible interaction. Transcriptomic surveys of insects in both interactions revealed that genes involved in cuticle formation, detoxification, metabolite transport, digestion, RNA processing, lipid or fatty acid metabolism, and proteolysis were significantly down-regulated during the incompatible interaction, whereas genes involved in insulin signaling were significantly upregulated. Knockdown of four genes, including the sugar transporter NlST45, the serine and arginine-rich protein NlSRp54, the cytochrome P450 gene NlCYP6AY1, and the cuticle protein NlCPR70 through RNA-interference revealed thess genes are important for BPH survival. Overall, the results of this study will be helpful for the future researches on BPH virulence shifts.