Mineral-based bulk structural materials (MBSMs) are known for their long history and extensive range of usage. The inherent brittleness of minerals poses a major problem to the performance of MBSMs. To overcome this problem, design principles have been extracted from natural biominerals, in which the extraordinary mechanical performance is achieved via the hierarchical organization of minerals and organics. Nevertheless, precise and efficient fabrication of MBSMs with bioinspired hierarchical structures under mild conditions has long been a big challenge. This Perspective provides a panoramic view of an emerging fabrication strategy, matrix-directed mineralization, which imitates the in vivo growth of some biominerals. The advantages of the strategy are revealed by comparatively analyzing the conventional fabrication techniques of artificial hierarchically structured MBSMs and the biomineral growth processes. By introducing recent advances, we demonstrate that this strategy can be used to fabricate artificial MBSMs with hierarchical structures. Particular attention is paid to the mass transport and the precursors that are involved in the mineralization process. We hope this Perspective can provide some inspiring viewpoints on the importance of biomimetic mineralization in material fabrication and thereby spur the biomimetic fabrication of high-performance MBSMs.