The Hall-Petch (HP) law, that accounts for the effect of grain size on the plastic yield stress of polycrystals, is revisited in terms of the collective motion of interacting dislocations. Sudden relaxation of incompatibility stresses in a grain triggers aftershocks in the neighboring ones. The HP law results from a scaling argument based on the conservation of the elastic energy during such transfers. The Hall-Petch law breakdown for nanometric sized grains is shown to stem from the loss of such a collective behavior as grains start deforming by successive motion of individual dislocations.