The aim of this article is to provide a systematic review of reliability studies of the sleep-wake disorder diagnostic criteria of the international classifications used in sleep medicine. Electronic databases (ubMed (1946-2021) and Web of Science (-2021)) were searched up to December 2021 for studies computing the Cohen's kappa coefficient of diagnostic criteria for the main sleep-wake disorder categories described in the principal classifications. Cohen's kappa coefficients were extracted for each main sleep-wake disorder category, for each classification subtype, and for the different types of methods used to test the degree of agreement about a diagnosis. The database search identified 383 studies. Fifteen studies were analyzed in this systematic review. Insomnia disorder (10/15) and parasomnia disorder (7/15) diagnostic criteria were the most studied. The reliability of all sleep-wake disorders presented a Cohen's kappa with substantial agreement (Cohen's kappa mean = 0.66). The two main reliability methods identified were "test-retest reliability" (11/15), principally used for International Classification of Sleep Disorders (ICSD), and "joint interrater reliability" (4/15), principally used for Diagnostic and Statistical Manual of Mental Disorders (DSM) subtype diagnostic criteria, in particularl, the DSM-5. The implications in terms of the design of the methods used to test the degree of agreement about a diagnosis in sleep medicine are discussed.
Keywords: Cohen’s kappa coefficient; clinical significance; field trial; reliability; sleep–wake disorders; systematic review.