Hydrogen production by electrocatalytic water splitting will play a key role in the realization of a sustainable energy supply. Owing to their relatively high stability and activity, iridium (hydr)oxides have been identified as the most promising catalysts for the oxidation of water. Comprehensive spectroscopic and theoretical studies on the basis of rutile IrO2 have provided insight about the electronic structure of the active X-ray amorphous phase. However, due to the absence of long-range order and missing information about the local arrangement of structural units, our present understanding of the active phase is very unsatisfying. In this work, using a combination of real-space atomic scale imaging with atomic pair distribution function analysis and local measurements of the electronic structure, we identify key structural motifs that are associated with high water splitting activity. Comparison of two X-ray amorphous phases with distinctively different electrocatalytic performance reveals that high activity is linked to the ratio between corner- and edge-sharing IrO6 octahedra. We show that the active and stable phase consists of single unit cell sized hollandite-like structural domains that are cross-linked through undercoordinated oxygen/iridium atoms. In the less active phase, the presence of the rutile phase structural motif results in a faster structural collapse and deactivation. The presented results provide insight into the structure-activity relationship and promote a rational synthesis of X-ray amorphous IrOx hydroxides that contain a favorable arrangement of structural units for improved performance in catalytic water splitting.