Recent studies have shown that head movements associated with cardiac activity contain a heart rate (HR) signal. In most previous studies, subjects were required to remain stationary in a specific environment during HR measurements, and measurement accuracy depended on the choice of target in the scene, i.e., the specified region of the face. In this paper, we proposed a robust HR measurement method based on ballistocardiogram (BCG) technology. This method requires only a camera and does not require that users establish a complex measurement environment. In addition, a bidirectional optical flow algorithm is designed to select and track valid feature points in the video captured by using the camera. Experiments with 11 subjects show that the HR values measured using the proposed method differ slightly from the reference values, and the average error is only 1.09%. Overall, this method can improve the accuracy of BCG without limitations related to skin tone, illumination, the state of the subject, or the test location.