Cell replacement therapy could represent an attractive alternative to insulin injections for the treatment of diabetes. However, this approach requires a thorough understanding of the molecular switches controlling the specification of the different pancreatic cell-types in vivo. These are derived from an apparently identical pool of cells originating from the early gut endoderm, which are successively specified towards the pancreatic, endocrine, and hormone-expressing cell lineages. Numerous studies have outlined the crucial roles exerted by transcription factors in promoting the cell destiny, defining the cell identity and maintaining a particular cell fate. This review focuses on the mechanisms regulating the morphogenesis of the pancreas with particular emphasis on recent findings concerning the transcription factor hierarchy orchestrating endocrine cell fate allocation.