Tandem mass tags (TMT) are one of the most widely used techniques in proteomics quantification due to their ability to accurately and precisely analyze up to 18 samples in a multiplexed manner. Moreover, TMT tags are introduced chemically by covalent coupling of the primary amines of digested proteins, making them universally applicable for any kind of sample. However, in addition to amine groups, the hydroxyl groups of serine, threonine, and tyrosine residues can also be labeled to some extent during TMT labeling, which compromises the analytical sensitivity and results in lower peptide identification rates compared to label-free methods. In this work, we investigated in-depth the chemical nature of TMT overlabeling and revealed that peptides simultaneously containing histidine and hydroxyl-containing residues were prone to overlabeling due to an intramolecular catalysis mediated by the histidyl imidazolyl group. Based on the understanding of the chemical mechanism, we developed a novel TMT labeling method under acidic pH that completely overcomes overlabeling. Compared to the standard labeling method provided by the TMT vendor, our method achieved comparable labeling efficiency on target groups but greatly reduced overlabeled peptides, resulting in the identification of 33.9% more unique peptides and 20.9% more proteins in proteomic analysis.