Although β-tricalcium phosphate (β-TCP) is widely used in clinical applications as a bone substitute owing to its positive tissue response and its ability to be replaced by new bone through a bone-remodeling process, it has the limitation of rapid resorption in vivo, which might become a reason for tissue atrophy and high crystallinity, which decrease biocompatibility. A reduction in the crystallinity might increase the biocompatibility of the bone substitute. To overcome the drawbacks of β-TCP, decrease in crystallinity and solubility, both are required. Therefore, in this study, the feasibility of fabricating Mg substituted low crystalline β-TCP (Mg-LC-β-TCP) granules formed in aqueous solution was evaluated in vivo focusing long-term adsorption and bone formation in bone defects formed in the rabbit femur using sintered β-TCP granules as a control. With Mg-LC-β-TCP, the resorption of the substitute was suppressed, and no tissue atrophy was observed even at 24 weeks post-implantation, whereas a few granules with surrounding tissue atrophy were observed at 12 weeks post-implantation. Tartrate-resistant acid phosphatase-staining indicated that the density of osteoclasts type cells with Mg-LC-β-TCP was significantly lower than that with β-TCP, and also the numbers of osteoblasts type cells with Mg-LC-β-TCP were significantly higher than that with β-TCP. It is suggested that Mg substitution to form low crystallinity β-TCP is a valuable way to overcome the limitations of β-TCP as a bone substitute.