SARS-CoV-2 causes profound changes in the sense of smell, including total smell loss. Although these alterations are often transient, many patients with COVID-19 exhibit olfactory dysfunction that lasts months to years. Although animal and human autopsy studies have suggested mechanisms driving acute anosmia, it remains unclear how SARS-CoV-2 causes persistent smell loss in a subset of patients. To address this question, we analyzed olfactory epithelial samples collected from 24 biopsies, including from nine patients with objectively quantified long-term smell loss after COVID-19. This biopsy-based approach revealed a diffuse infiltrate of T cells expressing interferon-γ and a shift in myeloid cell population composition, including enrichment of CD207+ dendritic cells and depletion of anti-inflammatory M2 macrophages. Despite the absence of detectable SARS-CoV-2 RNA or protein, gene expression in the barrier supporting cells of the olfactory epithelium, termed sustentacular cells, appeared to reflect a response to ongoing inflammatory signaling, which was accompanied by a reduction in the number of olfactory sensory neurons relative to olfactory epithelial sustentacular cells. These findings indicate that T cell-mediated inflammation persists in the olfactory epithelium long after SARS-CoV-2 has been eliminated from the tissue, suggesting a mechanism for long-term post-COVID-19 smell loss.