Gene expression, like many biological processes, is subject to noise. This noise has been measured on a global scale, but its general importance to the fitness of an organism is unclear. Here, I show that noise in gene expression in yeast has evolved to prevent harmful stochastic variation in the levels of genes that reduce fitness when their expression levels change. Therefore, there has probably been widespread selection to minimise noise in gene expression. Selection to minimise noise, because it results in gene expression that is stable to stochastic variation in cellular components, may also constrain the ability of gene expression to respond to non-stochastic variation. I present evidence that this has indeed been the case in yeast. I therefore conclude that gene expression noise is an important biological trait, and one that probably limits the evolvability of complex living systems.