We investigate how the degree-mixing pattern affects the emergence of cooperation in the networked prisoner's dilemma game. Our study shows that when a network becomes assortative mixing by degree, the large-degree vertices (hubs) tend to interconnect to each other closely, which destroys the sustainability among cooperators and promotes the invasion of defectors, whereas in disassortative networks, the isolation among hubs protects the cooperative hubs in holding onto their initial strategies to avoid extinction.