The slowly decaying viral dynamics, even after 2-3 weeks from diagnosis, is one of the characteristics of COVID-19 infection that is still unexplored in theoretical and experimental studies. This long-lived characteristic of viral infections in the framework of inherent variations or noise present at the cellular level is often overlooked. Therefore, in this work, we aim to understand the effect of these variations by proposing a stochastic non-Markovian model that not only captures the coupled dynamics between the immune cells and the virus but also enables the study of the effect of fluctuations. Numerical simulations of our model reveal that the long-range temporal correlations in fluctuations dictate the long-lived dynamics of a viral infection and, in turn, also affect the rates of immune response. Furthermore, predictions of our model system are in agreement with the experimental viral load data of COVID-19 patients from various countries.