The aim of this study is to investigate the impact of serum Mg on bone mineral metabolism in chronic kidney disease (CKD) patients with or without diabetes. A total of 56 CKD patients not receiving dialysis were recruited and divided into two groups, one group of 27 CKD patients with diabetes and another group of 29 CKD patients without diabetes. Biochemical determinations were made, and the estimated glomerular filtration rate (eGFR) was measured. Bone mineral density was measured by dual-energy X-ray absorptiometry. Serum Mg was inversely correlated with serum Ca (P = 0.023) and positively correlated with serum parathyroid hormone (PTH) (P = 0.020), alkaline phosphatase (P = 0.044), and phosphate (P = 0.040) in the CKD patients with diabetes. The CKD patients with diabetes had lower serum albumin and a higher proportion of hypomagnesemia and osteoporosis than the nondiabetic patients did (P < 0.05). Serum Mg was inversely correlated with eGFR in the CKD patients with or without diabetes (P < 0.05). Serum Mg showed an inverse correlation with 25-hydroxyvitamin D in CKD patients without diabetes (P = 0.006). Furthermore, the diabetic CKD patients with low serum Mg had a lower iPTH (P = 0.007) and a higher serum Ca/Mg ratio (P < 0.001) than the other CKD patients. The lower serum Mg subgroup showed a higher incidence of osteoporosis than the moderate and higher serum Mg subgroups did (66.7%, 39.4%, and 29.4%, resp.). In conclusion, low serum Mg may impact iPTH and exacerbates osteoporosis in CKD patients, particularly with diabetes.