The joint impact of pregnancy, environmental, and sociocultural exposures on early life gut microbiome is not yet well-characterized, especially in racially and socioeconomically diverse populations. Gut microbiota of 298 children from a Detroit-based birth cohort were profiled using 16S rRNA sequencing: 130 neonates (median age = 1.2 months) and 168 infants (median age = 6.6 months). Multiple factors were associated with neonatal gut microbiome composition in both single- and multi-factor models, with independent contributions of maternal race-ethnicity, breastfeeding, mode of delivery, marital status, exposure to environmental tobacco smoke, and indoor pets. These findings were consistent in the infants, and networks demonstrating the shared impact of factors on gut microbial composition also showed notable topological similarity between neonates and infants. Further, latent groups defined by these factors explained additional variation, highlighting the importance of combinatorial effects. Our findings also have implications for studies investigating the impact of the early life gut microbiota on disease.