We report the hierarchical structure of zirconium hydroxide after aging at different temperatures to elucidate the factors governing zirconium solubility in aqueous solutions. Zirconium hydroxide solid phases after aging at 25, 40, 60, and 90 °C under acidic to alkaline conditions were investigated using extended X-ray absorption fine structure (EXAFS), wide- and small-angle X-ray scattering (WAXS and SAXS), and transmission electron microscopy (TEM) techniques to reveal the bulk and surface structures of the solid phases from the nanoscale to sub-microscale. After aging at 25 °C, the fundamental building unit of the solid phase was considered to be tetrameric and dimeric hydroxide species. These polynuclear species formed amorphous primary particles that are approximately 3 nm in size, which in turn formed aggregates that are hundreds of nanometers in size. This hierarchical structure was found to be stable up to 60 °C under acidic and neutral conditions and up to 40 °C under alkaline conditions. After aging at 90 °C under acidic conditions and at 60 and 90 °C under alkaline conditions, the WAXS and EXAFS measurements suggested the crystallization of the solid phase. The SAXS profiles and TEM observations supported the existence of crystallized large particles about 60 nm in size, and the appearance of the Guinier region in the SAXS profiles indicated that the crystallization of the amorphous primary particles leads to the reduction of the size of the large aggregates. The transformation of the solid-phase structure by temperature was discussed in relation to the solubility product to understand the solubility-limiting solid phase. The solubility of zirconium hydroxide after aging at different temperatures was governed not only by the size of the amorphous primary particles or crystallized large particles but also by their surface configuration.