The origin and growth of magnetic fields in galaxies is still something of an enigma. It is generally assumed that seed fields are amplified over time through the dynamo effect, but there are few constraints on the timescale. It was recently demonstrated that field strengths as traced by rotation measures of distant (and hence ancient) quasars are comparable to those seen today, but it was unclear whether the high fields were in the unusual environments of the quasars themselves or distributed along the lines of sight. Here we report high-resolution spectra that demonstrate that the quasars with strong Mg II absorption lines are unambiguously associated with larger rotation measures. Because Mg ii absorption occurs in the haloes of normal galaxies along the sightlines to the quasars, this association requires that organized fields of surprisingly high strengths are associated with normal galaxies when the Universe was only about one-third of its present age.