Pituitary organogenesis is a highly complex and tightly regulated process that depends on several transcription factors (TFs), such as PROP1, PIT1 (POU1F1), HESX1, LHX3 and LHX4. Normal pituitary development requires the temporally and spatially organised expression of TFs and interactions between different TFs, DNA and TF co-activators. Mutations in these genes result in different combinations of hypopituitarism that can be associated with structural alterations of the central nervous system, causing the congenital form of panhypopituitarism. This review aims to elucidate the complex process of pituitary organogenesis, to clarify the role of the major TFs, and to compile the lessons learned from functional studies of TF mutations in panhypopituitarism patients and TF deletions or mutations in transgenic animals.