The introduction of fluorine (F) atoms onto conjugated polymer backbone has verified to be an effective way to enhance the overall performance of polymer-based bulk-heterojunction (BHJ) solar cells, but the underlying working principles are not yet fully uncovered. As our attempt to further understand the impact of F, herein we have reported two novel fluorinated analogues of PCDTBT, namely, PCDTFBT (1F) and PCDT2FBT (2F), through inclusion of either one or two F atoms into the benzothiadiazole (BT) unit of the polymer backbone and the characterization of their physical properties, especially their performance in solar cells. Together with a profound effect of fluorination on the optical property, nature of charge transport, and molecular organization, F atoms are effective in lowering both the HOMO and LUMO levels of the polymers without a large change in the energy bandgaps. PCDTFBT-based BHJ solar cell shows a power conversion efficiency (PCE) of 3.96 % with high open-circuit voltage (VOC) of 0.95 V, mainly due to the deep HOMO level (-5.54 eV). To the best of our knowledge, the resulting VOC is comparable to the record VOC values in single junction devices. Furthermore, to our delight, the best PCDTFBT-based device, prepared using 2 % v/v diphenyl ether (DPE) additive, reaches the PCE of 4.29 %. On the other hand, doubly-fluorinated polymer PCDT2FBT shows the only moderate PCE of 2.07 % with a decrease in VOC (0.88 V), in spite of the further lowering of the HOMO level (-5.67 eV) with raising the number of F atoms. Thus, our results highlight that an improvement in efficiency by tuning the energy levels of the polymers by means of molecular design can be expected only if their truly optimized morphologies with fullerene in BHJ systems are materialized.