Current biodiversity loss is mostly caused by anthropogenic habitat loss and fragmentation, climate change, and resource exploitation. Measuring the balance of species loss and gain in remaining fragmented landscapes throughout time entails a central research challenge. We resurveyed in 2013 plant species richness in the same plots of a previous sampling conducted in 2003 across 18 forest fragments of different sizes of the Chaco Serrano forest in Argentina. While the area of these forest remnants was kept constant, their surrounding forest cover changed over this time period. We compared plant species richness of both sampling years and calculated the proportion of species loss and gain at forest edges and interiors. As in 2003, we found a positive relationship between fragment area and plant richness in 2013 and both years showed a similar slope. However, we detected a net decrease of 24% of species' richness across all forest fragments, implying an unprecedentedly high rate and magnitude of species loss driven mainly by non-woody, short-lived species. There was a higher proportion of lost and gained species at forest edges than in forest interiors. Importantly, fragment area interacted with percent change in surrounding forest cover to explain the proportion of species lost. Small forest fragments showed a relatively constant proportion of species loss regardless of any changes in surrounding forest cover, whereas in larger fragments the proportion of species lost increased when surrounding forest cover decreased. We show that despite preserving fragment area, habitat quality and availability in the surroundings is of fundamental importance in shaping extinction and immigration dynamics of plant species at any given forest remnant. Because the Chaco Serrano forest has already lost 94% of its original cover, we argue that plant extinctions will continue through the coming decades unless active management actions are taken to increase native forest areas.