Unique thermal properties of metal clusters are believed to originate from the hierarchy of the bonding. However, an atomic-level understanding of how the bond stiffnesses are affected by the atomic packing of a metal cluster and the interfacial structure with the surrounding environment has not been attained to date. Here we elucidate the hierarchy in the bond stiffness in thiolate-protected, icosahedral-based gold clusters Au25(SC2H4Ph)18, Au38(SC2H4Ph)24 and Au144(SC2H4Ph)60 by analysing Au L3-edge extended X-ray absorption fine structure data. The Au-Au bonds have different stiffnesses depending on their lengths. The long Au-Au bonds, which are more flexible than those in the bulk metal, are located at the icosahedral-based gold core surface. The short Au-Au bonds, which are stiffer than those in the bulk metal, are mainly distributed along the radial direction and form a cyclic structural backbone with the rigid Au-SR oligomers.