In this work, we propose a predator-prey system with a Holling type Ⅱ functional response and study its dynamics when the prey exhibits vigilance behavior to avoid predation and predators exhibit cooperative hunting. We provide conditions for existence and the local and global stability of equilibria. We carry out detailed bifurcation analysis and find the system to experience Hopf, saddle-node, and transcritical bifurcations. Our results show that increased prey vigilance can stabilize the system, but when vigilance levels are too high, it causes a decrease in the population density of prey and leads to extinction. When hunting cooperation is intensive, it can destabilize the system, and can also induce bi-stability phenomenon. Furthermore, it can reduce the population density of both prey and predators and also change the stability of a coexistence state. We provide numerical experiments to validate our theoretical results and discuss ecological implications.
Keywords: bifurcation; global stability; hunting cooperation; vigilance.