Inhomogeneous temporal processes in natural and social phenomena have been described by bursts that are rapidly occurring events within short time periods alternating with long periods of low activity. In addition to the analysis of heavy-tailed interevent time distributions, higher-order correlations between interevent times, called correlated bursts, have been studied only recently. As the underlying mechanism behind such correlated bursts is far from being fully understood, we devise a simple model for correlated bursts using a self-exciting point process with a variable range of memory. Whether a new event occurs is stochastically determined by a memory function that is the sum of decaying memories of past events. In order to incorporate the noise and/or limited memory capacity of systems, we apply two memory loss mechanisms: a fixed number or a variable number of memories. By analysis and numerical simulations, we find that too much memory effect may lead to a Poissonian process, implying that there exists an intermediate range of memory effect to generate correlated bursts comparable to empirical findings. Our conclusions provide a deeper understanding of how long-range memory affects correlated bursts.