Energetic particle irradiation can cause surface ultra-smoothening, self-organized nanoscale pattern formation or degradation of the structural integrity of nuclear reactor components. A fundamental understanding of the mechanisms governing the selection among these outcomes has been elusive. Here we predict the mechanism governing the transition from pattern formation to flatness using only parameter-free molecular dynamics simulations of single-ion impacts as input into a multiscale analysis, obtaining good agreement with experiment. Our results overturn the paradigm attributing these phenomena to the removal of target atoms via sputter erosion: the mechanism dominating both stability and instability is the impact-induced redistribution of target atoms that are not sputtered away, with erosive effects being essentially irrelevant. We discuss the potential implications for the formation of a mysterious nanoscale topography, leading to surface degradation, of tungsten plasma-facing fusion reactor walls. Consideration of impact-induced redistribution processes may lead to a new design criterion for stability under irradiation.