In this Letter, we propose schemes to improve the performance of quantum batteries and provide a new, to the best of our knowledge, quantum source for a quantum battery without an external driving field. We show that the memory effect of the non-Markovian reservoir can play a significant role in improving the performance of quantum batteries, which originates from a backflow on the ergotropy in the non-Markovian regime, while there is no counterpart in Markovian approximation. We find that the peak for the maximum average storing power in the non-Markovian regime can be enhanced by manipulating the coupling strength between the charger and the battery. Finally, we find that the battery can also be charged by non-rotating wave terms without driving fields.