SARS-CoV-2 T cell responses are associated with COVID-19 recovery, and Class I- and Class II-restricted epitopes have been identified in the spike (S), nucleocapsid (N) and membrane (M) proteins and others. This prospective COVID-19 Health Action Response for Marines (CHARM) study enabled assessment of T cell responses against S, N and M proteins in symptomatic and asymptomatic SARS-CoV-2 infected participants. At enrollment all participants were negative by qPCR; follow-up occurred biweekly and bimonthly for the next 6 weeks. Study participants who tested positive by qPCR SARS-CoV-2 test were enrolled in an immune response sub-study. FluoroSpot interferon-gamma (IFN-γ) and IL2 responses following qPCR-confirmed infection at enrollment (day 0), day 7 and 14 and more than 28 days later were measured using pools of 17mer peptides covering S, N, and M proteins, or CD4+CD8 peptide pools containing predicted epitopes from multiple SARS-CoV-2 antigens. Among 124 asymptomatic and 105 symptomatic participants, SARS-CoV-2 infection generated IFN-γ responses to the S, N and M proteins that persisted longer in asymptomatic cases. IFN-γ responses were significantly (p = 0.001) more frequent to the N pool (51.4%) than the M pool (18.9%) among asymptomatic but not symptomatic subjects. Asymptomatic IFN-γ responders to the CD4+CD8 pool responded more frequently to the S pool (55.6%) and N pool (57.1%), than the M pool (7.1%), but not symptomatic participants. The frequencies of IFN-γ responses to the S and N+M pools peaked 7 days after the positive qPCR test among asymptomatic (S pool: 22.2%; N+M pool: 28.7%) and symptomatic (S pool: 15.3%; N+M pool 21.9%) participants and dropped by >28 days. Magnitudes of post-infection IFN-γ and IL2 responses to the N+M pool were significantly correlated with IFN-γ and IL2 responses to the N and M pools. These data further support the central role of Th1-biased cell mediated immunity IFN-γ and IL2 responses, particularly to the N protein, in controlling COVID-19 symptoms, and justify T cell-based COVID-19 vaccines that include the N and S proteins.