Reliability of Subjective Assessment of Spectral-Domain OCT Pathologic Features by Multiple Raters in Retinal Vein Occlusion

Sebastian Bemme; Amelie Heins; Peer Lauermann; Marcus Werner Storch; Mohammed Haitham Khattab; Hans Hoerauf; Nicolas Feltgen; Christian van Oterendorp

doi:10.1016/j.xops.2021.100031

Reliability of Subjective Assessment of Spectral-Domain OCT Pathologic Features by Multiple Raters in Retinal Vein Occlusion

Ophthalmol Sci. 2021 Jun 5;1(2):100031. doi: 10.1016/j.xops.2021.100031. eCollection 2021 Jun.

Authors

Sebastian Bemme¹, Amelie Heins¹, Peer Lauermann¹, Marcus Werner Storch¹, Mohammed Haitham Khattab¹, Hans Hoerauf¹, Nicolas Feltgen¹, Christian van Oterendorp¹

Affiliation

¹ Department of Ophthalmology, University Medical Center Göttingen, Göttingen, Germany.

Abstract

Purpose: To examine the interrater and intrarater reliability of qualitatively and quantitatively assessed disorganization of retinal inner layers (DRIL) and disorganization of retinal outer layers (DROL) by multiple raters. Subjectively assessing these surrogate biomarkers can be challenging in daily routine, despite the high resolution of spectral-domain (SD) OCT scans.

Design: Retrospective trial.

Participants: Three hundred six pooled SD OCT scans of 34 patients treated for macular edema caused by retinal vein occlusion (RVO) between January 2016 and December 2017.

Methods: SD OCT scans were assessed by 6 raters regarding presence of cystoid macular edema, subretinal fluid (SRF), vitreoretinal traction, and epiretinal membrane and extent of DRIL and DROL.

Main outcome measures: Interrater and intrarater reliability were calculated applying κ statistics for qualitative assessment regarding each pathologic feature's presence in all evaluated OCT scans, and for quantified horizontal DRIL and DROL extent within each OCT cross-section.

Results: Cystoid macular edema and SRF assessments revealed excellent inter- and intrarater reliability with almost perfect strength of agreement, whereas subjective DRIL and DROL evaluations yielded low κ statistics with slight to moderate strength of agreement. Furthermore, the presence of SRF remarkably compromised the reliability of DROL detection.

Conclusions: Our data highlight the limited subjective assessibility of DRIL and DROL, underscoring the need for automated image analysis to improve the reliability of OCT biomarkers for clinical studies and daily practice.

Keywords: CME, cystoid macular edema; DME, diabetic macular edema; DRIL; DRIL, disorganization of retinal inner layers; DROL; DROL, disorganization of retinal outer layers; ELM, external limiting membrane; ERM, epiretinal membrane; EZ, ellipsoid zone; IZ, interdigitation zone; RVO; RVO, retinal vein occlusion; SD, spectral-domain; SD-OCT; SRF, subretinal fluid; VRT, vitreoretinal traction; interrater reliability; κC, Cohen’s κ; κF, Fleiss’ κ.