Most highly pathogenic avian influenza A viruses cause only mild clinical signs in ducks, serving as an important natural reservoir of influenza A viruses. However, we isolated two H5N1 viruses that are genetically similar but differ greatly in virulence in ducks. A/Chicken/Jiangsu/k0402/2010 (CK10) is highly pathogenic, whereas A/Goose/Jiangsu/k0403/2010 (GS10) is low pathogenic. To determine the genetic basis for the high virulence of CK10 in ducks, we generated a series of single-gene reassortants between CK10 and GS10 and tested their virulence in ducks. Expression of the CK10 PA or hemagglutinin (HA) gene in the GS10 context resulted in increased virulence and virus replication. Conversely, inclusion of the GS10 PA or HA gene in the CK10 background attenuated the virulence and virus replication. Moreover, the PA gene had a greater contribution. We further determined that residues 101G and 237E in the PA gene contribute to the high virulence of CK10. Mutations at these two positions produced changes in virulence, virus replication, and polymerase activity of CK10 or GS10. Position 237 plays a greater role in determining these phenotypes. Moreover, the K237E mutation in the GS10 PA gene increased PA nuclear accumulation. Mutant GS10 viruses carrying the CK10 HA gene or the PA101G or PA237E mutation induced an enhanced innate immune response. A sustained innate response was detected in the brain rather than in the lung and spleen. Our results suggest that the PA and HA gene-mediated high virus replication and the intense innate immune response in the brain contribute to the high virulence of H5N1 virus in ducks.