Hexagonal boron nitride (h-BN) is a promising substrate for graphene based nano-electronic devices. We investigate the ballistic phonon transport at the interface of vertically stacked graphene and h-BN heterostructures using first principles density functional theory and atomistic Green's function simulations considering the influence of lattice stacking. We compute the frequency and wave-vector dependent transmission function and observe distinct stacking-dependent phonon transmission features for the h-BN/graphene/h-BN sandwiched systems. We find that the in-plane acoustic modes have the dominant contributions to the phonon transmission and thermal boundary conductance (TBC) for the interfaces with the carbon atom located directly on top of the boron atom (C-B matched) because of low interfacial spacing. The low interfacial spacing is a consequence of the differences in the effective atomic volume of N and B and the difference in the local electron density around N and B. For the structures with the carbon atom directly on top of the nitrogen atom (C-N matched), the spatial distance increases and the contribution of in-plane modes to the TBC decreases leading to higher contributions by out-of-plane acoustic modes. We find that the C-B matched interfaces have stronger phonon-phonon coupling than the C-N matched interfaces, which results in significantly higher TBC (more than 50%) in the C-B matched interface. The findings in this study will provide insights to understand the mechanism of phonon transport at h-BN/graphene/h-BN interfaces, to better explain the experimental observations and to engineer these interfaces to enhance heat dissipation in graphene based electronic devices.