The rapid growth of point-of-care polysomnographic alternatives has necessitated standardized evaluation and validation frameworks. The current average across participant validation methods may overestimate the agreement between wearable sleep tracker devices and polysomnography (PSG) systems because of the high base rate of sleep during the night and the interindividual difference across the sampling population. This study proposes an evaluation framework to assess the aggregating differences of the sleep architecture features and the chronologically epoch-by-epoch mismatch of the wearable sleep tracker devices and the PSG ground truth. An AASM-based sleep stage categorizing method was proposed to standardize the sleep stages scored by different types of wearable trackers. Sleep features and sleep stage architecture were extracted from the PSG and the wearable device's hypnograms. Therefrom, a localized quantifier index was developed to characterize the local mismatch of sleep scoring. We evaluated different commonly used wearable sleep tracking devices with the data collected from 22 different subjects over 30 nights of 8-h sleeping. The proposed localization quantifiers can characterize the chronologically localized mismatches over the sleeping time. The outperformance of the proposed method over existing evaluation methods was reported. The proposed evaluation method can be utilized for the improvement of the sensor design and scoring algorithm.
Keywords: polysomnography; sleep monitoring; sleep stage assessment; sleep stage scoring; wearable device.