The initiating genetic lesion in sporadically occurring cancers is impossible to identify. The existence of rare inherited cancer syndromes has helped to uncover some of the mutations that can initiate tumorigenesis. Most of these initiating lesions affect genes belonging to morphogenetic signaling pathways. We review the evidence that the cellular fate of individual epithelial cells in the adult is nonautonomous and depends on extrinsic information, just like cells in a developing embryo. Cancer stem cells need to disrupt these extrinsic restraints to gain an autonomous clonal proliferative advantage over neighboring stem cells.