T cells clear virus from the CNS and dynamically regulate brain functions, including spatial learning, through cytokine signaling. Here we determined whether hippocampal T cells that persist after recovery from infection with West Nile virus (WNV) or Zika virus (ZIKV) impact hippocampal-dependent learning and memory. Using newly established models of viral encephalitis recovery in adult animals, we show that in mice that have recovered from WNV or ZIKV infection, T cell-derived interferon-γ (IFN-γ) signaling in microglia underlies spatial-learning defects via virus-target-specific mechanisms. Following recovery from WNV infection, mice showed presynaptic termini elimination with lack of repair, while for ZIKV, mice showed extensive neuronal apoptosis with loss of postsynaptic termini. Accordingly, animals deficient in CD8+ T cells or IFN-γ signaling in microglia demonstrated protection against synapse elimination following WNV infection and decreased neuronal apoptosis with synapse recovery following ZIKV infection. Thus, T cell signaling to microglia drives post-infectious cognitive sequelae that are associated with emerging neurotropic flaviviruses.