Soil sustainability is reflected in a long-term balance between soil production and erosion for a given climate and geology. Here we evaluate soil sustainability in the Andean Altiplano where accelerated erosion has been linked to wetter climate from 4.5 ka and the rise of Neolithic agropastoralism in the millennium that followed. We measure in situ cosmogenic 14C directly on cultivated hilltops to quantify late Holocene soil loss, which we compare with background soil production rates determined from cosmogenic 26Al and 10Be. Our Monte Carlo-based inversion method identifies two scenarios to account for our data: an increase in erosion rate by 1-2 orders of magnitude between ~2.6 and 1.1 ka, or a discrete event stripping ~1-2 m of soil between ~1.9 and 1.1 ka. Coupled environmental and cultural factors in the Late Holocene signaled the onset of the pervasive human imprint in the Andean Altiplano seen today.