Glycated hemoglobin (HbA1c) is an indicator of the average blood glucose concentration. Failing to control HbA1c levels can accelerate the development of complications in patients with diabetes. Although metabolite profiles associated with HbA1c level in diabetes patients have been characterized using different platforms, more studies using high-throughput technology will be helpful to identify additional metabolites related to diabetes. Type 2 diabetes (T2D) patients were divided into two groups based on the HbA1c level: normal (HbA1c ≤6%) and high (HbA1c ≥9%) in both discovery and replication sets. A targeted metabolomics approach was used to quantify serum metabolites and multivariate logistic regression was used to identify significant differences between groups. The concentrations of 22 metabolites differed significantly between the two groups in the discovery set. In the replication set, the levels of 21 metabolites, including 16 metabolites identified in the discovery set, differed between groups. Among these, concentrations of eleven amino acids and one phosphatidylcholine (PC), lysoPC a C16:1, were higher and four metabolites, including three PCs (PC ae C36:1, PC aa C26:0, PC aa C34:2) and hexose, were lower in the group with normal HbA1c group than in the group with high HbA1c. Metabolites with high concentrations in the normal HbA1c group, such as glycine, valine, and PCs, may contribute to reducing HbA1c levels in patients with T2D. The metabolite signatures identified in this study provide insight into the mechanisms underlying changes in HbA1c levels in T2D.