Understanding and controlling the properties and dynamics of topological defects is a lasting challenge in the study of two-dimensional materials, and is crucial to achieve high-quality films required for technological applications. Here grain boundary structures, energies, and dynamics of binary two-dimensional materials are investigated through the development of a phase field crystal model that is parametrized to match the ordering, symmetry, energy, and length scales of hexagonal boron nitride. Our studies reveal some new dislocation core structures for various symmetrically and asymmetrically tilted grain boundaries, in addition to those obtained in previous experiments and first-principles calculations. We also identify a defect-mediated growth dynamics for inversion domains governed by the collective atomic migration and defect core transformation at grain boundaries and junctions, a process that is related to inversion symmetry breaking in binary lattice.