Pluripotent stem cells are characterised by continuous self-renewal while maintaining the potential to differentiate into cells of all three germ layers. Regulatory networks of maintaining pluripotency have been described in great detail and, similarly, there is great knowledge on key players that regulate their differentiation. Interestingly, pluripotency has various shades with distinct developmental potential, an observation that coined the term of a ground state of pluripotency. A precise interplay of signalling axes regulates ground state conditions and acts in concert with a combination of key transcription factors. The balance between these transcription factors greatly influences the integrity of the pluripotency network and latest research suggests that minute changes in their expression can strengthen but also collapse the network. Moreover, recent studies reveal different facets of these core factors in balancing a controlled and directed exit from pluripotency. Thereby, subsets of pluripotency-maintaining factors have been shown to adopt new roles during lineage specification and have been globally defined towards neuroectodermal and mesendodermal sets of embryonic stem cell genes. However, detailed underlying insights into how these transcription factors orchestrate cell fate decisions remain largely elusive. Our group and others unravelled complex interactions in the regulation of this controlled exit. Herein, we summarise recent findings and discuss the potential mechanisms involved.