Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks

Kun Zheng; Kangyi Ci; Hui Li; Lei Shao; Guangmin Sun; Junhua Liu; Jinling Cui

doi:10.1016/j.bspc.2022.103609

Heart rate prediction from facial video with masks using eye location and corrected by convolutional neural networks

Biomed Signal Process Control. 2022 May:75:103609. doi: 10.1016/j.bspc.2022.103609. Epub 2022 Mar 9.

Authors

Kun Zheng¹, Kangyi Ci¹, Hui Li¹, Lei Shao², Guangmin Sun¹, Junhua Liu¹, Jinling Cui¹

Affiliations

¹ Faculty of Information Technology, Beijing University of Technology, No.100, Pingleyuan, Chaoyang District, Beijing 100124, China.
² Department of Investigation, Sichuan Police College, No.186, Longtouguan Road, Jiangyang District, Luzhou, Sichuan 646000, China.

Abstract

Remote photoplethysmography (rPPG), which aims at measuring heart activities without any contact, has great potential in many applications. The emergence of novel coronavirus pneumonia COVID-19 has attracted worldwide attentions. Contact photoplethysmography (cPPG) methods need to contact the detection equipment with the patient, which may accelerate the spread of the epidemic. In the future, the non-contact heart rate detection will be an urgent need. However, existing heart rate measuring methods from facial videos are vulnerable to the less-constrained scenarios (e.g., with head movement and wearing a mask). In this paper, we proposed a method of heart rate detection based on eye location of region of interest (ROI) to solve the problem of missing information when wearing masks. Besides, a model to filter outliers based on residual network was conceived first by us and the better heart rate measurement accuracy was generated. To validate our method, we also created a mask dataset. The results demonstrated that after using our method for correcting the heart rate (HR) value measured with the traditional method, the accuracy reaches 4.65 bpm, which is 0.42 bpm higher than that without correction.

Keywords: Eye location; Heart rate; Measurement consistency; Residual network; rPPG.