It is widely believed that high-kinetic-energy (>1 keV) recoils in crystalline Si result in the formation of amorphous regions, whereas low-kinetic-energy recoils lead directly to isolated point defects. Here we study the response of a Si crystal using dynamical density-functional calculations and show that recoils of much less than 1 keV result in highly disordered regions that persist for hundreds of femtoseconds. Therefore, beam-induced defect formation is controlled by recrystallization processes during dynamic annealing even following low-energy ion implantation.