Functional brain network organization, measured by functional connectivity (FC), reflects key neurodevelopmental processes for healthy development. Early exposure to adversity, e.g. undernutrition, affects neurodevelopment, observable via disrupted FC, and leads to poorer outcomes from preschool age onward. We assessed longitudinally the impact of early growth trajectories on developmental FC in a rural Gambian population from age 5 to 24 months. To investigate how these early trajectories relate to later childhood outcomes, we assessed cognitive flexibility at 3-5 years. We observed that early physical growth before the fifth month of life drove optimal developmental trajectories of FC that in turn predicted cognitive flexibility at pre-school age. In contrast to previously studied developmental populations, this Gambian sample exhibited long-range interhemispheric FC that decreased with age. Our results highlight the measurable effects that poor growth in early infancy has on brain development and the subsequent impact on pre-school age cognitive development, underscoring the need for early life interventions throughout global settings of adversity.