Temporal changes in the Earth's magnetic field, known as geomagnetic secular variation, occur most prominently at low latitudes in the Atlantic hemisphere (that is, from -90 degrees east to 90 degrees east), whereas in the Pacific hemisphere there is comparatively little activity. This is a consequence of the geographical localization of intense, westward drifting, equatorial magnetic flux patches at the core surface. Despite successes in explaining the morphology of the geomagnetic field, numerical models of the geodynamo have so far failed to account systematically for this striking pattern of geomagnetic secular variation. Here we show that it can be reproduced provided that two mechanisms relying on the inner core are jointly considered. First, gravitational coupling aligns the inner core with the mantle, forcing the flow of liquid metal in the outer core into a giant, westward drifting, sheet-like gyre. The resulting shear concentrates azimuthal magnetic flux at low latitudes close to the core-mantle boundary, where it is expelled by core convection and subsequently transported westward. Second, differential inner-core growth, fastest below Indonesia, causes an asymmetric buoyancy release in the outer core which in turn distorts the gyre, forcing it to become eccentric, in agreement with recent core flow inversions. This bottom-up heterogeneous driving of core convection dominates top-down driving from mantle thermal heterogeneities, and localizes magnetic variations in a longitudinal sector centred beneath the Atlantic, where the eccentric gyre reaches the core surface. To match the observed pattern of geomagnetic secular variation, the solid material forming the inner core must now be in a state of differential growth rather than one of growth and melting induced by convective translation.