We study numerically the dynamics of a network of all-to-all-coupled, identical sub-networks consisting of diffusively coupled, non-identical FitzHugh-Nagumo oscillators. For a large range of within- and between-network couplings, the network exhibits a variety of dynamical behaviors, previously described for single, uncoupled networks. We identify a region in parameter space in which the interplay of within- and between-network couplings allows for a richer dynamical behavior than can be observed for a single sub-network. Adjoining this atypical region, our network of networks exhibits transitions to multistability. We elucidate bifurcations governing the transitions between the various dynamics when crossing this region and discuss how varying the couplings affects the effective structure of our network of networks. Our findings indicate that reducing a network of networks to a single (but bigger) network might not be accurate enough to properly understand the complexity of its dynamics.