Actigraphy-Based Sleep Detection: Validation with Polysomnography and Comparison of Performance for Nighttime and Daytime Sleep During Simulated Shift Work

Chenlu Gao; Peng Li; Christopher J Morris; Xi Zheng; Ma Cherrysse Ulsa; Lei Gao; Frank A J L Scheer; Kun Hu

doi:10.2147/NSS.S373107

Actigraphy-Based Sleep Detection: Validation with Polysomnography and Comparison of Performance for Nighttime and Daytime Sleep During Simulated Shift Work

Nat Sci Sleep. 2022 Oct 14:14:1801-1816. doi: 10.2147/NSS.S373107. eCollection 2022.

Authors

Chenlu Gao^{1

2

3}, Peng Li^{1

2

3}, Christopher J Morris^{1

2}, Xi Zheng^{1

3}, Ma Cherrysse Ulsa^{1

3}, Lei Gao^{1

2

3

4}, Frank A J L Scheer^{1

2

3}, Kun Hu^{1

2

3}

Affiliations

¹ Division of Sleep and Circadian Disorders, Departments of Medicine and Neurology, Brigham and Women's Hospital, Boston, MA, USA.
² Division of Sleep Medicine, Harvard Medical School, Boston, MA, USA.
³ Broad Institute of MIT and Harvard, Cambridge, MA, USA.
⁴ Department of Anesthesia, Critical Care and Pain Medicine, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA.

Abstract

Purpose: Actigraphy-based sleep detection algorithms were mostly validated using nighttime sleep, and their performance in detecting daytime sleep is unclear. We evaluated and compared the performance of Actiware and the Cole-Kripke algorithm (C-K) - two commonly used actigraphy-based algorithms - in detecting daytime and nighttime sleep.

Participants and methods: Twenty-five healthy young adults were monitored by polysomnography and actigraphy during two in-lab protocols with scheduled nighttime and/or daytime sleep (within-subject design). Mixed-effect models were conducted to compare the sensitivity, specificity, and F1 score (a less-biased measure of accuracy) of Actiware (with low/medium/high threshold setting, separately) and C-K in detecting sleep epochs from actigraphy recordings during nighttime/daytime. t-tests and intraclass correlation coefficients were used to assess the agreement between actigraphy-based algorithms and polysomnography in scoring total sleep time (TST).

Results: Sensitivity was similar between nighttime (Actiware: 0.93-0.99 across threshold settings; C-K: 0.61) and daytime sleep (Actiware: 0.93-0.99; C-K: 0.66) for both the C-K and Actiware (daytime/nighttime×algorithm interaction: p > 0.1). Specificity for daytime sleep was lower (Actiware: 0.35-0.54; C-K: 0.91) than that for nighttime sleep (Actiware: 0.37-0.62; C-K: 0.93; p = 0.001). Specificity was also higher for C-K than Actiware (p < 0.001), with no daytime/nighttime×algorithm interaction (p > 0.1). C-K had lower F1 (nighttime = 0.74; daytime = 0.77) than Actiware (nighttime = 0.95-0.98; daytime = 0.90-0.91) for both nighttime and daytime sleep (all p < 0.05). The daytime-nighttime difference in F1 was opposite for Actiware (daytime: 0.90-0.91; nighttime: 0.95-0.98) and C-K (daytime: 0.77; nighttime: 0.74; interaction p = 0.003). Bias in TST was lowest in Actiware (with medium-threshold) for nighttime sleep (underestimation of 5.99 min/8h) and in Actiware (with low-threshold) for daytime sleep (overestimation of 17.75 min/8h).

Conclusion: Daytime/nighttime sleep affected specificity and F1 but not sensitivity of actigraphy-based sleep scoring. Overall, Actiware performed better than the C-K algorithm. Actiware with medium-threshold was the least biased in estimating nighttime TST, and Actiware with low-threshold was the least biased in estimating daytime TST.

Keywords: Actiware; Cole-Kripke algorithm; circadian rhythms; shift worker; sleep scoring.

Grants and funding

This research was supported by NIH R01HL094806, RF1AG064312, RF1AG059867. C.G. and P.L. are also supported by the BrightFocus Foundation Alzheimer’s Disease Research Program (A2020886S). C.G. is additionally supported by the Alzheimer’s Association (AARFD-22-928372). L.G. is also supported by the National Institute on Aging (NIA) grant (R03AG067985). F.A.J.L.S. was further supported by NIH R01HL153969.