The plethora of recent discoveries in the field of topological electronic insulators has inspired a search for boson systems with similar properties. There are predictions that ferromagnets on a two-dimensional honeycomb lattice may host chiral edge magnons. In such systems, we theoretically study how magnons and phonons couple. We find topological magnon polarons around the avoided crossings between phonons and topological magnons. Exploiting this feature along with our finding of Rayleigh-like edge phonons in armchair ribbons, we demonstrate the existence of chiral edge modes with a phononic character. We predict that these modes mediate a chirality in the coherent phonon response and suggest measuring this effect via elastic transducers. These findings reveal a possible approach towards heat management in future devices.