Biological and technological systems process information by means of cascades of signals. Be they interacting genes, spiking neurons or electronic transistors, information travels across these systems, producing, for each set of external conditions, an appropriate response. In technology, circuits performing specific complex tasks are designed by humans. In biology, however, design has to be ruled out, confronting us with the question of how these systems could have arisen by accumulation of small changes. The key factor is the genotype-phenotype map. With the exception of RNA folding, not much is known about the exact nature of this mapping. Here, we show that structure of the genotype-phenotype map of simple feed-forward circuits is very close to the ones found in RNA; they have a large degree of neutrality, by which a circuit can be completely rewired keeping its input-output function intact, and there is a relatively small neighbourhood of a given circuit containing almost all the phenotypes.