Why metalloenzymes often show dramatic changes in their catalytic activity when subjected to chemically similar but non-native metal substitutions is a long-standing puzzle. Here, we report on the catalytic roles of metal ions in a model metalloenzyme system, human carbonic anhydrase II (CA II). Through a comparative study on the intermediate states of the zinc-bound native CA II and non-native metal-substituted CA IIs, we demonstrate that the characteristic metal ion coordination geometries (tetrahedral for Zn2+, tetrahedral to octahedral conversion for Co2+, octahedral for Ni2+, and trigonal bipyramidal for Cu2+) directly modulate the catalytic efficacy. In addition, we reveal that the metal ions have a long-range (~10 Å) electrostatic effect on restructuring water network in the active site. Our study provides evidence that the metal ions in metalloenzymes have a crucial impact on the catalytic mechanism beyond their primary chemical properties.