Both ice and silica crystallize into solid-state structures composed of tetrahedral building units that are joined together to form an infinite four-connected net. Mathematical considerations suggest that there is a vast number of such nets and thus potential crystal structures. It is therefore perhaps surprising to discover that, despite the differences in the nature of interatomic interactions in these materials, a fair number of commonly observed ice and silica phases are based on common nets. Here we use computer simulation to investigate the origin of this symmetry between the structures formed for ice and silica and to attempt to understand why it is not complete. We start from a comparison of the dense phases and then move to the relationship between the different open (zeolitic and clathratic) structures formed for both materials. We show that there is a remarkably strong correlation between the energetics of isomorphic silica and water ice structures and that this correlation arises because of the strong link between the total energy of a material and its local geometric features. Finally, we discuss a number of as yet unsynthesized low-energy structures which include a phase of ice based on quartz, a silica based on the structure of ice VI, and an ice clathrate that is isomorphic to the silicate structure nonasil.