Momentum diffusion is a widespread phenomenon in generic Hamiltonian systems. We show for the prototypical standard map that this implies weak ergodicity breaking for the superdiffusive transport in coordinate direction with an averaging-dependent quadratic and cubic increase of the mean-squared displacement (MSD), respectively. This is explained via integrated Brownian motion, for which we derive aging time dependent expressions for the ensemble-averaged MSD, the distribution of time-averaged MSDs, and the ergodicity breaking parameter. Generalizations to other systems showing momentum diffusion are pointed out.