Phase separation at nanoscale and highly dispersive nanoparticles were exploited to fabricate a novel type of nanocrystalline W-Cu-Cr-ZrC composite with special hierarchical structure. The microstructures were characterized elaborately and the formation mechanisms of the hierarchical structure were disclosed. It was found that the supersaturated Cr separated from W during sintering and segregated at profuse interfaces. Moreover, Cr can also interact with ZrC nanoparticles dispersed in W matrix, leading to the formation of Zr-Cr-C phase which exerts significant effect on inhibiting coarsening of W grains. On the other side, the dissolution of Cr and W contributes to the formation of Cu nanocrystalline structure. As a result, the prepared W-Cu-Cr-ZrC composite exhibits an ultrahigh hardness of 943HV owing to the synergy of nanocrystalline structure and dispersion strengthening of nanoparticles. The hardness has achieved the highest value among the W-Cu-based composites with the same Cu content reported in literature. This study provided a new strategy for production of high-performance W-Cu-based composites through combination of microstructural design with addition of doping element and nanoparticles.