Boron strongly modifies electronic and diffusion properties of graphite. We report the first ab initio study of boron interaction with the point defects in graphite, which includes structures, thermodynamics, and diffusion. A number of possible diffusion mechanisms of boron in graphite are suggested. We conclude that boron diffuses in graphite by a kick-out mechanism. This mechanism explains the common activation energy, but large magnitude difference, for the rate of boron diffusion parallel and perpendicular to the basal plane.