With increasing demand on a laser source in the gigahertz pulse repetition rate regime, clarification on the mechanism of instability in high repetition rate fiber lasers - a promising alternative to solid state lasers - is of great importance and can potentially offer guideline for continuous wave (CW) mode locking. Here we present a theoretical approach together with relevant experimental corroboration to analyze the instabilities. By means of appropriate approximations, regimes from Q-switched mode locking, CW mode locking and pulsation are theoretically identified. Meanwhile, a critical curve that characterizes pump level for triggering Q-switched mode locking and pulsation for different repetition rates is given by virtue of both analytical and numerical procedures. In experiment, a passively mode-locked fiber laser with 1.6 GHz fundamental repetition rate is realized. The three regimes and corresponding pump power intervals are revealed, which are in consistence with theoretical prediction. Pulsation, as a relatively exotic phenomenon in GHz fiber laser, is well reproduced by the present model, which further verifies the accuracy of the approach as well as enriches the nonlinear dynamics.