A photonic-assisted frequency downconverter with self-interference cancellation and fiber dispersion elimination is proposed for in-band full-duplex (IBFD) radio-over-fiber (ROF) systems based on stimulated Brillouin scattering. In this work, a dual-polarization Mach-Zehnder modulator (DPol-MZM) is employed to cancel the self-interference signal in the optical domain, thus the proposed system has a large operation bandwidth without the limitation of the electrical bottleneck. Meanwhile, a widely tunable microwave photonic filter (MPF) based on stimulated Brillouin scattering (SBS) is constructed to perform carrier-suppressed single-sideband modulation of LO. Therefore, the proposed photonic frequency downconverter is naturally free from fiber dispersion. Furthermore, thanks to the characteristics of the SBS-based MPF, such as large out-of-band rejection ratio, high resolution and wideband tunability, the proposed system has a high flexibility and downconversion efficiency, which is of great significance for IBFD ROF systems.