Bone marrow mesenchymal stem cells (BMSCs), the important component and regulator of bone marrow microenvironment, give rise to hematopoietic-supporting stromal cells and form hematopoietic niches for hematopoietic stem cells (HSCs). However, how BMSC differentiation affects hematopoiesis is poorly understood. In this review, we focus on the role of BMSC differentiation in hematopoiesis. We discussed the role of BMSCs and their progeny in hematopoiesis. We also examine the mechanisms that cause differentiation bias of BMSCs in stress conditions including aging, irradiation, and chemotherapy. Moreover, the differentiation balance of BMSCs is crucial to hematopoiesis. We highlight the negative effects of differentiation bias of BMSCs on hematopoietic recovery after bone marrow transplantation. Keeping the differentiation balance of BMSCs is critical for hematopoietic recovery. This review summarises current understanding about how BMSC differentiation affects hematopoiesis and its potential application in improving hematopoietic recovery after bone marrow transplantation.