An asteroid's history is determined in large part by its strength against collisions with other objects1,2 (impact strength). Laboratory experiments on centimetre-scale meteorites3 have been extrapolated and buttressed with numerical simulations to derive the impact strength at the asteroid scale4,5. In situ evidence of impacts on boulders on airless planetary bodies has come from Apollo lunar samples6 and images of the asteroid (25143) Itokawa7. It has not yet been possible, however, to assess directly the impact strength, and thus the absolute surface age, of the boulders that constitute the building blocks of a rubble-pile asteroid. Here we report an analysis of the size and depth of craters observed on boulders on the asteroid (101955) Bennu. We show that the impact strength of metre-sized boulders is 0.44 to 1.7 megapascals, which is low compared to that of solid terrestrial materials. We infer that Bennu's metre-sized boulders record its history of impact by millimetre- to centimetre-scale objects in near-Earth space. We conclude that this population of near-Earth impactors has a size frequency distribution similar to that of metre-scale bolides and originates from the asteroidal population. Our results indicate that Bennu has been dynamically decoupled from the main asteroid belt for 1.75 ± 0.75 million years.