The reconstruction of osteochondral tissue remains a challenging task in clinical therapy because of its heterogeneous structure. The best way to face the challenge is to develop a biomimetic construct to mimic the multilayered gradient from cartilage, to calcified cartilage and subchondral bone. In this study, bone marrow mesenchymal stromal cells (BMSCs) were cultured on electrospun fibrous meshes and cell sheets were incubated. The fibrous meshes were composed of 50% poly(L-lactide) and 50% gelatin, displaying excellent biocompatibility, cell affinity and degradability. Differentiation of BMSC sheets on fibrous meshes was induced chondrogenically or osteogenically. In particular, the BMSC sheets were able to be efficiently induced differentiating towards calcified cartilage by using a 1:1 (v/v) mixed medium of chondrogenic and osteogenic inductive media. Thus, a gradient 3D construct was built by stacking the differently pre-differentiated cell/mesh complexes layer by layer from top to bottom to mimic the cartilage-to-bone transition. With this gradient construct being implanted in the rabbit knee osteochondral defect, it was confirmed that it could promote the tissue regeneration with intact cartilage layer formation in comparison with the multilayered construct without a gradient. The strategy of using properly pre-differentiated BMSC sheet on fibrous mesh to build the osteochondral interface was thus suggested as being feasible and effective in mimicking its hierarchical complexity, and favored the repairing of injured joint cartilage.