The benefits of self-monitoring of blood glucose (SMBG) on glycemic control among type 2 diabetes (T2DM) patients not receiving insulin remains controversial. This study aimed to examine the association between SMBG and glycemic control in these patients. This retrospective longitudinal study enrolled 4987 eligible patients from a medical center in Taiwan. Data were collected from electronic medical records at 0 (baseline), 3, 6, 9, and 12 (end-point) months after enrollment. Patients were assigned to the early SMBG group or to the non-user group depending on whether they performed SMBG at baseline. Differences in glycated hemoglobin (HbA1c) reduction between groups at each time-point were assessed using SMBG group-by-time interaction in generalized estimating equations models, which were established using backward elimination method for multivariate regression analysis. Subgroup analyses for patients using non-insulin and insulin secretagogues were performed additionally. The estimated maximal difference in HbA1c reduction between groups (early SMBG users vs. non-users) was 0.55% at 3 months. Subgroup analyses showed maximal differences of 0.61% and 0.52% at 3 months in the non-insulin and insulin secretagogues groups, respectively. SMBG group-by-time interaction was statistically significant at 3 months and lasted for 12 months. The finding suggests that performing SMBG at disease onset was positively associated with better glycemic control in newly diagnosed non-insulin-treated T2DM patients, regardless whether non-insulin secretagogues or insulin secretagogues were used.