As a highly promising clean energy source to replace fossil fuels in the 21st century, hydrogen energy has garnered considerable attention, with water electrolysis emerging as a key hydrogen production technology. The development of highly active and stable non-precious metal-based catalysts for the hydrogen evolution reaction (HER) is crucial for achieving efficient and low-cost hydrogen production through electrolysis. Recently, heterostructure composite catalysts comprising two or more non-precious metals have demonstrated outstanding catalytic performance. First, we introduced the basic mechanism of the HER and, based on the reported HER theory, discussed the essence of constructing heterostructures to improve the catalytic activity of non-noble metal-based catalysts, that is, the coupling effect between components effectively regulates the electronic structure and the position of d-band centers. Then three catalytic effects of non-precious metal-based heterogeneous catalysts are described: synergistic effect, electron transfer effect and support effect. Lastly, we emphasized the potential of non-precious metal-based heterogeneous catalysts to replace precious metal-based catalysts, and summarized the future prospects and challenges.