Prospective cohort study to evaluate the accuracy of sleep measurement by consumer-grade smart devices compared with polysomnography in a sleep disorders population

Abstract

Objectives: Consumer-grade smart devices are now commonly used by the public to measure waking activity and sleep. However, the ability of these devices to accurately measure sleep in clinical populations warrants more examination. The aim of the present study was to assess the accuracy of three consumer-grade sleep monitors compared with gold standard polysomnography (PSG).

Design: A prospective cohort study was performed.

Setting: Adults undergoing PSG for investigation of a suspected sleep disorder.

Participants: 54 sleep-clinic patients were assessed using three consumer-grade sleep monitors (Jawbone UP3, ResMed S+ and Beddit) in addition to PSG.

Outcomes: Jawbone UP3, ResMed S+ and Beddit were compared with gold standard in-laboratory PSG on four major sleep parameters-total sleep time (TST), sleep onset latency (SOL), wake after sleep onset (WASO) and sleep efficiency (SE).

Results: The accelerometer Jawbone UP3 was found to overestimate TST by 28 min (limits of agreement, LOA=-100.23 to 157.37), with reasonable agreement compared with gold standard for TST, WASO and SE. The doppler radar ResMed S+ device underestimated TST by 34 min (LOA=-257.06 to 188.34) and had poor absolute agreement compared with PSG for TST, SOL and SE. The mattress device, Beddit underestimated TST by 53 min (LOA=-238.79 to 132) on average and poor reliability compared with PSG for all measures except TST. High device synchronisation failure occurred, with 20% of recordings incomplete due to Bluetooth drop out and recording loss.

Conclusion: Poor to moderate agreement was found between PSG and each of the tested devices, however, Jawbone UP3 had relatively better absolute agreement than other devices in sleep measurements compared with PSG. Consumer grade devices assessed do not have strong enough agreement with gold standard measurement to replace clinical evaluation and PSG sleep testing. The models tested here have been superseded and newer models may have increase accuracy and thus potentially powerful patient engagement tools for long-term sleep measurement.

Keywords: information technology; respiratory medicine (see thoracic medicine); sleep medicine; telemedicine.