The Mamyshev oscillator (MO) can generate high-performance pulses. However, due to their non-resonant cavities, they usually are not self-starting, and there is almost no effort to reveal the pulse buildup dynamics of the MO. This paper investigates the dynamic of single pulse (SP) and multi-pulse formation in a self-starting MO. It indicated that both SP self-starting and multi-pulse self-starting can be obtained by adjusting the oscillator parameters. More importantly, increasing pump power could only result in bound state pulses (BSPs) if SP self-starting was formed. With the increase of the pump power, the pulse number in BSPs would increase. However, multiple pulses could not be formed only by increasing the pump power, and the BSPs obtained here underwent SP generated from noise, amplified, and then bounded, which is different from conventional passive mode-locked fiber lasers (CPMLFLs). On the other hand, if multiple pulses were self-initiated, BSPs, pulse bunch, and harmonic mode-locked pulses (HMLPs) could be obtained by adjusting the polarization state and pump power in the cavity. Furthermore, once any of the above states are formed, if the oscillator polarization state and filter interval are unchanged, only increasing the pump power from zero, the original state can still be obtained, which is consistent with the characteristics of the CPMLFLs. These findings will provide new insights into the pulse dynamics of self-starting MO, which will be significant for studying ultrafast laser technology and nonlinear optics.