Recently, a sparse adaptive algorithm termed zero-attracting sign least-mean-square (ZA-SLMS), has been clarified to be able to reconstruct robustly heartbeat spectrum by Doppler radar signal. However, since the strengths of noise evidently differ under different body motions, the sparse heartbeat spectra cannot be always acquired accurately by the constant regularization parameter (REPA) that balances the gradient correction and the sparse penalty, applying in the ZA-SLMS algorithm. In this paper, an improved ZA-SLMS algorithm is proposed by introducing adaptive REPA (AREPA), where the proportion of sparse penalty is adjusted based on the standard deviation of radar data. Moreover, to enhance the stability of heartbeat detection, a time-window-variation (TWV) technique is further introduced in the improved ZA-SLMS algorithm, considering the fact that the position of spectral peak associated with the heart rate (HR) is stable when the length of time window changes within a short period. Experimental results measured against five subjects validated that our proposal reliably improves the error of HR estimation than the standard ZA-SLMS algorithm.