We report on first-principles calculations of the side-jump contribution to the anomalous Hall conductivity (AHC) directly from the electronic structure of a perfect crystal. We implemented our approach for a short-range scattering disorder model within the density functional theory and computed the full scattering-independent AHC in elemental bcc Fe, hcp Co, fcc Ni, and L1(0) FePd and FePt alloys. The full AHC thus calculated agrees systematically with experiment to a degree unattainable so far, correctly capturing the previously missing elements of side-jump contributions, hence paving the way to a truly predictive theory of the anomalous Hall effect and turning it from a characterization tool to a probing tool of multiband complex electronic band structures.