During development, pluripotency is a transient state describing a cell's ability to give rise to all three germ layers and germline. Recent studies have shown that, in vitro, pluripotency is highly dynamic: exogenous stimuli provided to cultures of mouse embryonic stem cells, isolated from pre-implantation blastocysts, significantly affect the spectrum of pluripotency. 2i/LIF, a recently defined serum-free medium, forces mouse embryonic stem cells into a ground-state of pluripotency, while serum/LIF cultures promote the co-existence of ground-like and primed-like mouse embryonic stem cell subpopulations. The latter heterogeneity correlates with temporal fluctuations of pluripotency markers, including the master regulator Nanog, in single cells. We propose a mathematical model of Nanog dynamics in both media, accounting for recent experimental data showing the persistence of a small Nanog Low subpopulation in ground-state pluripotency mouse embryonic stem cell cultures. The model integrates into the core pluripotency Gene Regulatory Network both inhibitors present in 2i/LIF (PD and Chiron), and feedback interactions with genes found to be differentially expressed in the two media. Our simulations and bifurcation analysis show that, in ground-state cultures, Nanog dynamics result from the combination of reduced noise in gene expression and the shift of the system towards a monostable, but still excitable, regulation. Experimental data and agent-based modelling simulations indicate that mouse embryonic stem cell proliferation dynamics vary in the two media, and cannot be reproduced by accounting only for Nanog-dependent cell-cycle regulation. We further demonstrate that both PD and Chiron play a key role in regulating heterogeneity in transcription factor expression and, ultimately, mouse embryonic stem cell fate decision.