Early excessive weight gain is positively associated with later obesity, and yet the effect of weight gain during specific periods and the impact of infant feeding practices are debated. The objective of the present study was to examine the impact of weight gain in periods of early childhood on body composition at 3 years, and whether infant feeding modified the relationship between early growth and body composition at 3 years. We studied 233 children from the prospective cohort study, SKOT (in Danish: Småbørns Kost og Trivsel). Birth weight z-scores (BWZ) and change in weight-for-age z-scores (WAZ) from 0 to 5, 5 to 9, 9 to 18 and 18 to 36 months were analysed for relations with body composition (anthropometry and bioelectrical impedance) at 3 years by multivariate regression analysis. BWZ and change in WAZ from 0 to 5 months were positively associated with BMI, fat mass index (FMI) and fat-free mass index (FFMI) at 3 years. Full breastfeeding for 6 months (compared to less than 1 month) eliminated the effect of early growth (P = 0.01). Full breastfeeding for 6 months (compared to less than 1 month) also eliminated the positive relation between BWZ and FMI (P = 0.009). No effect modification of infant feeding was found for FFMI. In conclusion, high birth weight and rapid growth from 0 to 5 months were associated with increased FMI and FFMI at 3 years. Longer duration of full breastfeeding reduced the effect of birth weight and early weight gain on fat mass.