Harmful environmental stimuli during critical stages of development can profoundly affect behavior and susceptibility to diseases. Alzheimer disease (AD) is the most frequent neurodegenerative disease, and evidence suggest that inflammatory conditions act cumulatively, contributing to disease onset. Here we investigated whether infection early in life can contribute to synapse damage and cognitive impairment induced by amyloid-β oligomers (AβOs), neurotoxins found in AD brains. To this end, wild-type mice were subjected to neonatal (post-natal day 4) infection by Escherichia coli (1 × 104 CFU/g), the main cause of infection in low-birth-weight premature infants in the US. E. coli infection caused a transient inflammatory response in the mouse brain starting shortly after infection. Although infected mice performed normally in behavioral tasks in adulthood, they showed increased susceptibility to synapse damage and memory impairment induced by low doses of AβOs (1 pmol; intracerebroventricular) in the novel object recognition paradigm. Using in vitro and in vivo approaches, we show that microglial cells from E. coli-infected mice undergo exacerbated activation when exposed to low doses of AβOs. In addition, treatment of infected pups with minocycline, an antibiotic that inhibits microglial pro-inflammatory polarization, normalized microglial response to AβOs and restored normal susceptibility of mice to oligomer-induced cognitive impairment. Interestingly, mice infected with by E. coli (1 × 104 CFU/g) during adolescence (post-natal day 21) or adulthood (post-natal day 60) showed normal cognitive performance even in the presence of AβOs (1 pmol), suggesting that only infections at critical stages of development may lead to increased susceptibility to amyloid-β-induced toxicity. Altogether, our findings suggest that neonatal infections can modulate microglial response to AβOs into adulthood, thus contributing to amyloid-β-induced synapse damage and cognitive impairment.