The Mpemba effect is a counterintuitive relaxation phenomenon whereby a system with a higher initial temperature may cool down to the thermal state faster than an identical system that was initially prepared at a lower temperature. Here, we investigate heat and work in a Markovian state transition system with cyclic switching hot-cold temperatures, which operates as an Otto heat engine working in long but finite time, either with or without the Mpemba effect. Under the condition of the periodic steady state having been reached, the time durations of the heating and cooling relaxation processes are determined by exploring a distance-from-equilibrium equivalent to the Kullback-Leibler divergence. We then numerically evaluate and compare the averages and variances of both the work and the power output of two scenarios with and without the Mpemba effect. The results show that the Markovian Mpemba effect can enhance the machine performance by significantly increasing the power output for a given efficiency without sacrificing the stability.