Gene networks typically involve the regulatory control of multiple genes with related function. This connectivity enables correlated control of the levels and timing of gene expression. Here we study how gene expression timing in the single-input module motif can be encoded in the regulatory DNA of a gene. Using stochastic simulations, we examine the role of binding affinity, TF regulatory function and network size in controlling the mean first-passage time to reach a fixed fraction of steady-state expression for both an auto-regulated TF gene and a target gene. We also examine how the variability in first-passage time depends on these factors. We find that both network size and binding affinity can dramatically speed up or slow down the response time of network genes, in some cases predicting more than a 100-fold change compared to that for a constitutive gene. Furthermore, these factors can also significantly impact the fidelity of this response. Importantly, these effects do not occur at "extremes" of network size or binding affinity, but rather in an intermediate window of either quantity.