Currently, qigong and tai chi exercises are the two most common preventive as well as therapeutic interventions for chronic metabolic diseases such as type 2 diabetes mellitus (T2DM). However, the quantitative evaluation of these interventions is limited. This study aimed to evaluate the therapeutic efficacy of qigong and tai chi intervention in middle-aged and older adults with T2DM. The study included 103 eligible participants, who were randomized to participate for 12 weeks, in one of the following intervention groups for the treatment of T2DM: fitness qigong, tai chi, and control group. Three biochemical measures, including fasting plasma glucose (FPG), glycated hemoglobin (HbA1C), and C-peptide (C-P) levels, assessed at baseline and 12 weeks, served as the primary outcome measures. During the training process, 16 of the 103 participants dropped out. After the 12-week intervention, there were significant influences on HbA1C (F2,83 = 4.88, p = 0.010) and C-P levels (F2,83 = 3.64, p = 0.031). Moreover, significant reduction in C-P levels was observed after 12-week tai chi practice (p = 0.004). Furthermore, there was a significant negative correlation between the duration of T2DM and the relative changes in FPG levels after qigong intervention, and the relative changes in HbA1C levels were positively correlated with waist-to-height ratio after tai chi practice. Our study suggests that targeted qigong exercise might have a better interventional effect on patients with a longer duration of T2DM, while tai chi might be risky for people with central obesity. Trial registration: This trial was registered in Chinese Clinical Trial Registry. The registration number is ChiCTR180020069. The public title is "Health-care qigong · study for the prescription of chronic diabetes intervention."