We have carried out detailed magnetic and transport studies of the new Sr0.5Ce0.5FBiS2-xSex (0.0 ≤ x ≤ 1.0) superconductors derived by doping Se in Sr0.5Ce0.5FBiS2. Se-doping produces several effects: it suppresses semiconducting-like behavior observed in the undoped Sr0.5Ce0.5FBiS2, the ferromagnetic ordering temperature, TFM, decreases considerably from 7.5 K (in Sr0.5Ce0.5FBiS2) to 3.5 K and the superconducting transition temperature, Tc, gets enhanced slightly to 2.9-3.3 K. Thus in these Se-doped materials, TFM is marginally higher than Tc. Magnetization studies provide evidence of bulk superconductivity in Sr0.5Ce0.5FBiS2-xSex at x ≥ 0.5 in contrast to the undoped Sr0.5Ce0.5FBiS2 (x = 0) where magnetization measurements indicate a small superconducting volume fraction. Quite remarkably, as compared with the effective paramagnetic Ce-moment (~2.2 μB), the ferromagnetically ordered Ce-moment in the superconducting state is rather small (~0.1 μB) suggesting itinerant ferromagnetism. To the best of our knowledge, Sr0.5Ce0.5FBiS2-x Sex (x = 0.5 and 1.0) are distinctive Ce-based bulk superconducting itinerant ferromagnetic materials with Tc < TFM. Furthermore, a novel feature of these materials is that they exhibit a dual and quite unusual hysteresis loop corresponding to both the ferromagnetism and the coexisting bulk superconductivity.