We report the first observation of the magnon-polariton bistability in a cavity magnonics system consisting of cavity photons strongly interacting with the magnons in a small yttrium iron garnet (YIG) sphere. The bistable behaviors emerged as sharp frequency switchings of the cavity magnon polaritons (CMPs) and related to the transition between states with large and small numbers of polaritons. In our experiment, we align, respectively, the [100] and [110] crystallographic axes of the YIG sphere parallel to the static magnetic field and find very different bistable behaviors (e.g., clockwise and counter-clockwise hysteresis loops) in these two cases. The experimental results are well fitted and explained as being due to the Kerr nonlinearity with either a positive or negative coefficient. Moreover, when the magnetic field is tuned away from the anticrossing point of CMPs, we observe simultaneous bistability of both magnons and cavity photons by applying a drive field on the lower branch.