Heart rate variability is one of major physiological parameters to reflect our stress, which has motivated researchers to investigate a Doppler sensor-based non-contact heartbeat interval estimation algorithm. As one of such methods, we have previously proposed a spectrogram-based method. In this method, the spectrum that might be due to heartbeats is integrated over a spectrogram, and then heartbeat interval is estimated by detecting peaks over the integrated spectrum. However, when a subject moves, undesired peaks with large amplitude appear, which causes the incorrect peak detection. As one of the technique to eliminate the undesired peaks with large amplitude, there is CA-CFAR (Cell Average-Constant False Alarm Rate). CA-CFAR is the technique to detect a signal, when the amplitude of a signal exceeds a threshold calculated with average amplitude of signals before and after the investigated one. However, depending on the duration of body movements, the influence of body movements might be included within the signals used for the threshold calculation, which might results in the detection failure of undesired peaks. This is because the length of GT (Guard Time) is fixed, where GT is the time to prevent the signal used for the threshold calculation from including the investigated signal components. To solve this problem, we propose a novel CA-CFAR in which the length of GT is set as the latest peak interval and only the signal before the investigated one is used so that the influence of body movements does not affect the threshold calculation. Through the experiments where a subject moves, i.e., typing, we confirmed that our spectrogram-based heart rate variability estimation method with the proposed CA-CFAR outperformed the one with CA-CFAR based on fixed GT by the RMSE (Root Mean Squared Error) between the estimated heartbeat interval and the ground truth value of the one.