We find that discrete noise of inhibiting (signal) molecules can greatly delay the extinction of plasmids in a plasmid replication system: a prototypical biochemical regulatory network. We calculate the probability distribution of the metastable state of the plasmids and show in this example that the reaction rate equations may fail in predicting the average number of regulated molecules even when this number is large, and the time is much shorter than the mean extinction time.