Serum carcinoembryonic antigen (CEA) levels can help predict the prognosis of colorectal cancer patients. Accordingly, high preoperative CEA levels that is not restored after surgery are indicative of a worse outcome. On the other hand, smoking can increase serum CEA levels independently of the disease status. Thus, we aimed to evaluate the impact of smoking on the prognostic value of serum CEA levels. This retrospective cohort study included 273 patients who underwent curative resection for stage I-III colorectal adenocarcinoma at a single institution, between January 2010 and December 2017. Patients were grouped as follows: group A, normal preoperative and postoperative CEA levels (n = 152); group B, elevated preoperative CEA levels that returned to reference values after surgery (n = 69); and group C, elevated postoperative serum CEA levels (n = 52). Patients were also grouped according to their smoking history: group S (current smokers, n = 79) and group NS (never and former smokers, n = 194). Group A showed a higher 3-year disease-free survival (DFS) rate (84.9%) than groups B (75.4%) and C (62.0%) (p < 0.001). Postoperative serum CEA levels were significantly higher in the S group than in the NS group (2.6 vs. 3.1 ng/mL, p = 0.009), whereas preoperative levels were similar (3.8 vs. 4.1, p = 0.182). Further, smokers showed higher 3 year-DFS rates than nonsmokers in group C (83.3% vs. 43.9%, p = 0.029). This suggests that while elevated postoperative CEA levels are associated with lower DFS rates in never and former smokers, they are not associated with lower DFS rates in current smokers. We conclude that persistent smoking alters the prognostic value of postoperative serum CEA levels in colorectal cancer patients and that, consequently, alternative surveillance strategies need to be developed for colon cancer patients with smoking habits.