Multiple-band degeneracy has been widely recognized to be beneficial for high thermoelectric performance. Here, we discover that the p-type Dirac bands with lower degeneracy synergistically produce a higher Seebeck coefficient and electrical conductivity in topological semimetal BaAgBi. The anomalous transport phenomenon intrinsically originated from the asymmetric electronic structures: (i) complete p-type Dirac bands near the Fermi level facilitate high and strong energy-dependent hole relaxation time; (ii) the presence of additional parabolic conduction valleys allows for a large density of states to accept scattered electrons, leading to an enlarged hole-electron relaxation time ratio and, thus, weakened bipolar effect. In combination with the strong lattice anharmonicity, an exceptional p-type average ZT of 0.42 is achieved from 300 to 600 K, which can be dramatically enhanced to 1.38 via breaking the C3v symmetry. This work uncovers the underlying mechanisms governing the abnormal transport behavior in Dirac semimetal BaAgBi and highlights the asymmetric electronic structures as target features to discover/design high-performance thermoelectric materials.