The interferon-induced transmembrane (IFITM) protein family is a group of antiviral restriction factors that impair flexibility and inhibit membrane fusion at the plasma or the endosomal membrane, restricting viral progression at entry. While IFITMs are widely known to inhibit several single-stranded RNA viruses, there are limited reports available regarding their effect in double-stranded DNA viruses. In this work, we have analyzed a possible antiviral function of IFITMs against a double stranded DNA virus, the African swine fever virus (ASFV). Infection with cell-adapted ASFV isolate Ba71V is IFN sensitive and it induces IFITMs expression. Interestingly, high levels of IFITMs caused a collapse of the endosomal pathway to the perinuclear area. Given that ASFV entry is strongly dependent on endocytosis, we investigated whether IFITM expression could impair viral infection. Expression of IFITM1, 2 and 3 reduced virus infectivity in Vero cells, with IFITM2 and IFITM3 having an impact on viral entry/uncoating. The role of IFITM2 in the inhibition of ASFV in Vero cells could be related to impaired endocytosis-mediated viral entry and alterations in the cholesterol efflux, suggesting that IFITM2 is acting at the late endosome, preventing the decapsidation stage of ASFV.