The uncertainty relation is a fundamental limit in quantum mechanics and is of great importance to quantum information processing as it relates to quantum precision measurement. Due to interactions with the surrounding environment, a quantum system will unavoidably suffer from decoherence. Here, we investigate the dynamic behaviors of the entropic uncertainty relation of an atom-cavity interacting system under a bosonic reservoir during the crossover between Markovian and non-Markovian regimes. Specifically, we explore the dynamic behavior of the entropic uncertainty relation for a pair of incompatible observables under the reservoir-induced atomic decay effect both with and without quantum memory. We find that the uncertainty dramatically depends on both the atom-cavity and the cavity-reservoir interactions, as well as the correlation time, τ, of the structured reservoir. Furthermore, we verify that the uncertainty is anti-correlated with the purity of the state of the observed qubit-system. We also propose a remarkably simple and efficient way to reduce the uncertainty by utilizing quantum weak measurement reversal. Therefore our work offers a new insight into the uncertainty dynamics for multi-component measurements within an open system, and is thus important for quantum precision measurements.