Novel anti-HIV agents are still needed to overcome resistance issues, in particular inhibitors acting against novel viral targets. The ribonuclease H (RNase H) function of the reverse transcriptase (RT) represents a validated and promising target, and no inhibitor has reached the clinical pipeline yet. Here, we present rationally designed non-diketo acid selective RNase H inhibitors (RHIs) based on the quinolinone scaffold starting from former dual integrase (IN)/RNase H quinolinonyl diketo acids. Several derivatives were synthesized and tested against RNase H and viral replication and found active at micromolar concentrations. Docking studies within the RNase H catalytic site, coupled with site-directed mutagenesis, and Mg2+ titration experiments demonstrated that our compounds coordinate the Mg2+ cofactor and interact with amino acids of the RNase H domain that are highly conserved among naïve and treatment-experienced patients. In general, the new inhibitors influenced also the polymerase activity of RT but were selective against RNase H vs the IN enzyme.