Early detection of atherosclerosis is important for patients with type 2 diabetes mellitus because cardiovascular disease (CVD) is a main cause of death in these people. In this study, we investigated the relationship between an arterial stiffness parameter called cardio-ankle vascular index (CAVI) and coronary artery calcification (CAC). We performed a cross-sectional study in 371 type 2 diabetic patients with clinical suspicion of coronary heart disease (CHD). We evaluated the relationships between CAVI and CAC score determined by multislice computed tomography as well as major cardiovascular risk factors, including age, body mass index, hemoglobinA1c and the Framingham CHD risk score. CAVI was correlated with age (r = 0.301, p < 0.0001), uric acid (r = 0.236, p < 0.0001), estimated glomerular filtration rate (r = -0.145, p = 0.0166), CHD risk score (r = 0.327, p < 0.0001) and log (CAC + 1) (r = 0.303, p < 0.0001). The area under the receiver operating characteristic curve for CAVI was higher than that of CHD risk score in predicting CAC >0, CAC >100, CAC >400, or CAC >1000. CAVI is positively correlated with CAC, and is considered to be a useful method to detect CAC.