Bornean orang-utans experienced a major demographic decline and local extirpations during the Pleistocene and Holocene due to climate change, the arrival of modern humans, of farmers and recent commercially-driven habitat loss and fragmentation. The recent loss of habitat and its dramatic fragmentation has affected the patterns of genetic variability and differentiation among the remaining populations and increased the extinction risk of the most isolated ones. However, the contribution of recent demographic events to such genetic patterns is still not fully clear. Indeed, it can be difficult to separate the effects of recent anthropogenic fragmentation from the genetic signature of prehistoric demographic events. Here, we investigated the genetic structure and population size dynamics of orang-utans from different sites. Altogether 126 individuals were analyzed and a full-likelihood Bayesian approach was applied. All sites exhibited clear signals of population decline. Population structure is known to generate spurious bottleneck signals and we found that it does indeed contribute to the signals observed. However, population structure alone does not easily explain the observed patterns. The dating of the population decline varied across sites but was always within the 200-2000 years period. This suggests that in some sites at least, orang-utan populations were affected by demographic events that started before the recent anthropogenic effects that occurred in Borneo. These results do not mean that the recent forest exploitation did not leave its genetic mark on orang-utans but suggests that the genetic pool of orang-utans is also impacted by more ancient events. While we cannot identify the main cause for this decline, our results suggests that the decline may be related to the arrival of the first farmers or climatic events, and that more theoretical work is needed to understand how multiple demographic events impact the genome of species and how we can assess their relative contributions.