HIV-1 specific cellular immune responses play a significant part in controlling HIV-1 viral replication and are an important component of an HIV-1 vaccine induced immune response. We reported earlier that recombinant DNA vaccine delivered intramuscularly, and recombinant Listeria monocytogenes, delivered orally induced CD8+ and CD4+ T cell immune responses in rhesus macaques and that this vaccine protocol showed partial protection against an SIV239 challenge. In this paper, we have analyzed the SIV antigen-specific immune responses at the time of challenge and during the subsequent infection course. We find that the immune status of the animals, as measured by the frequency of antigen-specific IFN-gamma secreting peripheral blood mononuclear cells, at the time of challenge correlates more strongly with viral loads at set point than peak viral loads. The correlation between the immune response and viral load was strongest early, as viral set-point was just being established and disintegrates overtime. This study demonstrates the cellular immune response to SIV at the time of challenge of a nonhuman primate is able to impact on viral set-point following SIV239 challenge. Further, this study demonstrates that as virus replicates the T cell immune response to SIV antigens induced by the vaccine is modulated by antigen encountered by immune cells during viral replication.