Research has shown that high-dose supplemental dietary fiber intake has beneficial effects on cardiovascular risk factors. To clarify such a relationship, we examined the association between daily dietary fiber intake and plasma lipids using a cross-sectional design including 1034 (M 502, F 532) rural-to-urban workers in China. We found a dose-response relationship between increased dietary fiber intakes and increase of HDL cholesterol in male workers. There was also a dose-response relationship between increased dietary fiber intake and decreased total cholesterol to HDL cholesterol (TC/HDL-C) ratio in both male and female workers, after adjusting for potential confounders (p for trend, all p < 0.05). When the average dietary fiber intake increased from less than 18 g/day to over 30 g/day, the average HDL cholesterol level increased by 10.1%, and the TC/HDL-C ratio decreased by 14.4% for males (p = 0.020) and by 11.1% for females (p = 0.048). In conclusion, higher daily dietary fiber consumption is associated with beneficial effect on cholesterol for rural-to-urban workers in China, suggesting its potential beneficial effect on decreasing the risk of cardiovascular diseases.