Embryonic stem cells (ESCs) and induced pluripotent stem cells (iPSCs) are characterized by their unique capacity to stepwise differentiate towards any particular cell type in an adult organism. Pluripotent stem cells provide a beneficial platform to model hereditary diseases and even cancer development. While the incidence of pancreatic diseases such as diabetes and pancreatitis is increasing, the understanding of the underlying pathogenesis of particular diseases remains limited. Only a few recent publications have contributed to the characterization of human pancreatic development in the fetal stage. Hence, most knowledge of pancreatic specification is based on murine embryology. Optimizing and understanding current in vitro protocols for pancreatic differentiation of ESCs and iPSCs constitutes a prerequisite to generate functional pancreatic cells for better disease modeling and drug discovery. Moreover, human pancreatic organoids derived from pluripotent stem cells, organ-restricted stem cells, and tumor samples provide a powerful technology to model carcinogenesis and hereditary diseases independent of genetically engineered mouse models. Herein, we summarize recent advances in directed differentiation of pancreatic organoids comprising endocrine cell types. Beyond that, we illustrate up-and-coming applications for organoid-based platforms.