Hyperacute rejection, mediated by natural anti-Galalpha1,3Galbeta1,4GlcNAc (alphaGal) antibodies and the classically activated complement pathway, was identified as the first major barrier to the survival of porcine organs in humans. Subsequently, discordant pig-to-nonhuman primate and concordant rodent models revealed key roles for T and B lymphocytes in the second form of rejection, acute vascular rejection (AVR) or delayed xenograft rejection (DXR). As significant progress was made in strategies to circumvent or suppress xenoreactivity of the adaptive immune system, it became clear that, apart from natural antibodies, other innate immune system elements actively participate in AVR/DXR and represent a barrier to xenograft acceptance that may be particularly difficult to overcome. Observations in pig-to-primate and semi-discordant and concordant rodent models indicate that Natural Killer (NK) cells play a more prominent role in xenograft than in allograft rejection. Several mechanisms through which human NK cells recognize porcine endothelial cells have been elucidated and these appear to be more diverse than those involved in NK cell alloreactivity. Further, it has been demonstrated that human macrophages and neutrophils can directly recognize pig derived cells and can mediate direct xenograft damage. Here, we review the recent progress in the understanding of the xenoreactivity of the natural immune system, focussing on preclinical pig-to-(non)human primate systems, and discuss the proposed strategies to overcome these barriers.