Cadherin trafficking controls tissue morphogenesis and cell polarity. The endocytic adaptor Numb participates in apicobasal polarity by acting on intercellular adhesions in epithelial cells. However, it remains largely unknown how Numb controls cadherin-based adhesion. Here, we found that Numb directly interacted with p120 catenin (p120), which is known to interact with E-cadherin and prevent its internalization. Numb accumulated at intercellular adhesion sites and the apical membrane in epithelial cells. Depletion of Numb impaired E-cadherin internalization, whereas depletion of p120 accelerated internalization. Expression of the Numb-binding fragment of p120 inhibited E-cadherin internalization in a dominant-negative fashion, indicating that Numb interacts with the E-cadherin/p120 complex and promotes E-cadherin endocytosis. Impairment of Numb induced mislocalization of E-cadherin from the lateral membrane to the apical membrane. Atypical protein kinase C (aPKC), a member of the PAR complex, phosphorylated Numb and inhibited its association with p120 and α-adaptin. Depletion or inhibition of aPKC accelerated E-cadherin internalization. Wild-type Numb restored E-cadherin internalization in the Numb-depleted cells, whereas a phosphomimetic mutant or a mutant with defective α-adaptin-binding ability did not restore the internalization. Thus, we propose that aPKC phosphorylates Numb to prevent its binding to p120 and α-adaptin, thereby attenuating E-cadherin endocytosis to maintain apicobasal polarity.