The entanglement produced by a bilinear Hamiltonian in continuous variables has been thoroughly studied and widely used. In contrast, the physics of entanglement resulting from nonlinear interaction described by partially degenerate high-order Hamiltonians remains unclear. Here, we derive a hierarchy of sufficient and necessary conditions for the positive-partial-transposition separability of bipartite nonlinear quantum states. The proposed criteria detect the nonpositive-partial-transposition inseparability of higher-order moments of states, which provides a systematic framework for the characterization of this kind of entanglement. Through numerical simulation of cubic and quartic Hamiltonians, we demonstrate the existence and competition of a hierarchy of entanglement witnesses, revealing the mechanism underlying such entanglement. Our results may provide a new direction in continuous variable quantum information processing.