SARS-CoV-2 primarily replicates in mucosal sites, and more information is needed about immune responses in infected tissues. Here, we used rhesus macaques to model protective primary immune responses in tissues during mild COVID-19. Viral RNA levels were highest on days 1-2 post-infection and fell precipitously thereafter. 18F-fluorodeoxyglucose (FDG)-avid lung abnormalities and interferon (IFN)-activated monocytes and macrophages in the bronchoalveolar lavage (BAL) were found on days 3-4 post-infection. Virus-specific effector CD8+ and CD4+ T cells became detectable in the BAL and lung tissue on days 7-10, after viral RNA, radiologic evidence of lung inflammation, and IFN-activated myeloid cells had substantially declined. Notably, SARS-CoV-2-specific T cells were not detectable in the nasal turbinates, salivary glands, and tonsils on day 10 post-infection. Thus, SARS-CoV-2 replication wanes in the lungs of rhesus macaques prior to T cell responses, and in the nasal and oral mucosa despite the apparent lack of antigen-specific T cells, suggesting that innate immunity efficiently restricts viral replication during mild COVID-19.