African Swine Fever virus (ASFV) is a large double-enveloped DNA virus of the Asfarviridae family that causes a lethal hemorrhagic disease in domestic pigs and wild boars. Since 2007, a highly virulent genotype II strain has emerged and spread in Europe and South-East Asia, where millions of animals succumbed to the disease. Field- and laboratory-attenuated strains of ASFV cause highly variable clinical disease severity and survival, and mechanisms remain unclear. We hypothesized that the immunological and hygienic status of pigs is a determinant of ASF disease course. Here we compared the immunological profile at baseline and in response to ASFV infection in specific pathogen-free (SPF) and farm-raised Large White domestic pigs. At steady state, SPF pigs showed lower white blood cell counts and a lower basal inflammatory and antiviral transcriptomic profile compared to farm pigs, associated with profound differences in gut microbiome composition. After inoculation with a highly virulent ASFV genotype II strain (Armenia 2008), severe clinical signs, viremia and pro-inflammatory cytokines appeared sooner in SPF pigs, indicating a reduced capacity to control early virus replication. In contrast, during infection with an attenuated field isolate (Estonia 2014), SPF pigs presented a milder and shorter clinical disease with full recovery, whereas farm pigs presented severe protracted disease with 50% lethality. Interestingly, farm pigs showed higher production of inflammatory cytokines, whereas SPF pigs produced more anti-inflammatory IL-1ra early after infection and presented a stronger expansion of leukocytes in the recovery phase. Altogether, our data indicate that the hygiene-dependent innate immune status has a double-edge sword impact on immune responses in ASF pathogenesis. While the higher baseline innate immune activity helps the host in reducing initial virus replication, it promotes immunopathological cytokine responses, and delays lymphocyte proliferation after infection with an attenuated strain. Such effects should be considered for live vaccine development and vigilance.