The entire decaying dynamics of harmonic mode-locking (HML) are studied utilizing the dispersive Fourier transform (DFT) technique in a SESAM-based mode-locked fiber laser. It is unveiled that the harmonic solitons do not disappear directly, but undergo transitional processes from the higher-order HML to the lower-order HML and then to the fundamental mode-locking (FML), and finally vanish. The "big corner" can also exist in the decaying process rather than just in the buildup process of HML, and there is at least one "big corner" during the decaying process between the consecutive multi-pulsing states. The energy stabilization phase (ESP) cannot be observed during every transitional process. A breathing behavior and a vibrating soliton molecule are observed in the decaying process from the 2nd HML to the FML and in the decaying process of the FML, respectively. Our work would enrich the understanding of HML behaviors and may contribute to the laser designs.