We study the excitation conditions of the supported field modes, as well as the spontaneous decay property of a two-level quantum emitter coupled to photonic structures containing topological insulators (TIs) and left-handed materials. Within the proper field quantization scheme, the spontaneous decay rates of dipoles with different polarizations are expressed in forms of the Green's functions. We find that in the proposed structure, the variation in the topological magnetoelectric polarizability (TMP) has a deterministic effect on the excitation of different field modes. As the result, the spontaneous decay property of the quantum emitter can be engineered. For a dipole placed in different spatial regions, the spontaneous decay feature indicates a dominant contribution from the waveguide modes, the surface plasmon modes or the free vacuum modes. Moreover, a special kind of the surface plasmon modes displaying asymmetric density of states at the interfaces, becomes legal in the presence of nontrivial TIs. These phenomena manifest the feasibility in controlling dipole emissions via manipulations of the topological magnetoelectric (TME) effect. Our results have potential applications in quantum technologies relied on the accurate control over light-matter interactions.