We theoretically investigate the localization mechanism of the quantum anomalous Hall effect (QAHE) in the presence of spin-flip disorders. We show that the QAHE stays quantized at weak disorders, then enters a Berry-curvature mediated metallic phase at moderate disorders, and finally goes into the Anderson insulating phase at strong disorders. From the phase diagram, we find that at the charge neutrality point although the QAHE is most robust against disorders, the corresponding metallic phase is much easier to be localized into the Anderson insulating phase due to the interchange of Berry curvatures carried, respectively, by the conduction and valence bands. In the end, we provide a phenomenological picture related to the topological charges to better understand the underlying physical origin of the QAHE Anderson localization.