The importance of highly siderophile elements (HSEs; namely, gold, iridium, osmium, palladium, platinum, rhenium, rhodium and ruthenium) in tracking the late accretion stages of planetary formation has long been recognized. However, the precise nature of the Moon's accretional history remains enigmatic. There is a substantial mismatch in the HSE budgets of the Earth and the Moon, with the Earth seeming to have accreted disproportionally more HSEs than the Moon1. Several scenarios have been proposed to explain this conundrum, including the delivery of HSEs to the Earth by a few big impactors1, the accretion of pebble-sized objects on dynamically cold orbits that enhanced the Earth's gravitational focusing factor2, and the 'sawtooth' impact model, with its much reduced impact flux before about 4.10 billion years ago3. However, most of these models assume a high impactor-retention ratio (the fraction of impactor mass retained on the target) for the Moon. Here we perform a series of impact simulations to quantify the impactor-retention ratio, followed by a Monte Carlo procedure considering a monotonically decaying impact flux4, to compute the impactor mass accreted into the lunar crust and mantle over their histories. We find that the average impactor-retention ratio for the Moon's entire impact history is about three times lower than previously estimated1,3. Our results indicate that, to match the HSE budgets of the lunar crust and mantle5,6, the retention of HSEs should have started 4.35 billion years ago, when most of the lunar magma ocean was solidified7,8. Mass accreted before this time must have lost its HSEs to the lunar core, presumably during lunar mantle crystallization9. The combination of a low impactor-retention ratio and a late retention of HSEs in the lunar mantle provides a realistic explanation for the apparent deficit of the Moon's late-accreted mass relative to that of the Earth.