Water molecules can arrange into a liquid with complex hydrogen-bond networks and at least 17 experimentally confirmed ice phases with enormous structural diversity. It remains a puzzle how or whether this multitude of arrangements in different phases of water are related. Here we investigate the structural similarities between liquid water and a comprehensive set of 54 ice phases in simulations, by directly comparing their local environments using general atomic descriptors, and also by demonstrating that a machine-learning potential trained on liquid water alone can predict the densities, lattice energies, and vibrational properties of the ices. The finding that the local environments characterising the different ice phases are found in water sheds light on the phase behavior of water, and rationalizes the transferability of water models between different phases.