Assessing the Accuracy of Popular Commercial Technologies That Measure Resting Heart Rate and Heart Rate Variability

Jason D Stone; Hana K Ulman; Kaylee Tran; Andrew G Thompson; Manuel D Halter; Jad H Ramadan; Mark Stephenson; Victor S Finomore Jr; Scott M Galster; Ali R Rezai; Joshua A Hagen

doi:10.3389/fspor.2021.585870

Assessing the Accuracy of Popular Commercial Technologies That Measure Resting Heart Rate and Heart Rate Variability

Front Sports Act Living. 2021 Mar 1:3:585870. doi: 10.3389/fspor.2021.585870. eCollection 2021.

Authors

Jason D Stone¹, Hana K Ulman^{1

2}, Kaylee Tran^{1

3}, Andrew G Thompson¹, Manuel D Halter¹, Jad H Ramadan¹, Mark Stephenson^{1

4}, Victor S Finomore Jr¹, Scott M Galster¹, Ali R Rezai¹, Joshua A Hagen¹

Affiliations

¹ Rockefeller Neuroscience Institute, West Virginia University, Morgantown, WV, United States.
² Department of Chemical and Biomedical Engineering, West Virginia University, Morgantown, WV, United States.
³ College of Arts and Sciences, Boston University, Boston, MA, United States.
⁴ National Football League, Detroit Lions, Detroit, MI, United States.

Abstract

Commercial off-the shelf (COTS) wearable devices continue development at unprecedented rates. An unfortunate consequence of their rapid commercialization is the lack of independent, third-party accuracy verification for reported physiological metrics of interest, such as heart rate (HR) and heart rate variability (HRV). To address these shortcomings, the present study examined the accuracy of seven COTS devices in assessing resting-state HR and root mean square of successive differences (rMSSD). Five healthy young adults generated 148 total trials, each of which compared COTS devices against a validation standard, multi-lead electrocardiogram (mECG). All devices accurately reported mean HR, according to absolute percent error summary statistics, although the highest mean absolute percent error (MAPE) was observed for CameraHRV (17.26%). The next highest MAPE for HR was nearly 15% less (HRV4Training, 2.34%). When measuring rMSSD, MAPE was again the highest for CameraHRV [112.36%, concordance correlation coefficient (CCC): 0.04], while the lowest MAPEs observed were from HRV4Training (4.10%; CCC: 0.98) and OURA (6.84%; CCC: 0.91). Our findings support extant literature that exposes varying degrees of veracity among COTS devices. To thoroughly address questionable claims from manufacturers, elucidate the accuracy of data parameters, and maximize the real-world applicative value of emerging devices, future research must continually evaluate COTS devices.

Keywords: electrocardiogram; heart rate; heart rate variability; photoplethysmography; root mean square of successive differences; validation; wearables.