Structured Reporting of Head and Neck Sonography Achieves Substantial Interrater Reliability

Benjamin Philipp Ernst; Carla Dörsching; Alessandro Bozzato; Jennis Gabrielpillai; Sven Becker; Matthias Frank Froelich; Benedikt Kramer; Christoph Sproll; Mirco Schapher; Miguel Goncalves; Naglaa Mansour; Benedikt Hofauer; Wieland H Sommer; Felix von Scotti; Johannes Matthias Weimer; Julian Künzel

doi:10.1055/a-2173-3966

Structured Reporting of Head and Neck Sonography Achieves Substantial Interrater Reliability

Ultrasound Int Open. 2023 Oct 5;9(1):E26-E32. doi: 10.1055/a-2173-3966. eCollection 2023 Jan.

Authors

Benjamin Philipp Ernst¹, Carla Dörsching¹, Alessandro Bozzato², Jennis Gabrielpillai¹, Sven Becker³, Matthias Frank Froelich⁴, Benedikt Kramer⁵, Christoph Sproll⁶, Mirco Schapher⁷, Miguel Goncalves⁸, Naglaa Mansour⁹, Benedikt Hofauer¹⁰, Wieland H Sommer¹¹, Felix von Scotti¹², Johannes Matthias Weimer¹³, Julian Künzel¹⁴

Affiliations

¹ Department of Otorhinolaryngology, University Medical Center Bonn, Bonn, Germany.
² Department of Otorhinolaryngology, Head & Neck Surgery, Saarland University Hospital and Saarland University Faculty of Medicine, Homburg, Germany.
³ Department of Otorhinolaryngology, Head and Neck Surgery, University of Tübingen Medical Center, Tuebingen, Germany.
⁴ Department of Radiology and Nuclear Medicine, University Medical Centre Mannheim, Mannheim, Germany.
⁵ Department of Otorhinolaryngology, Head and Neck Surgery, University Medical Centre Mannheim, Mannheim, Germany.
⁶ Department of Oral and Maxillofacial Surgery, Medical Faculty and University Hospital Düsseldorf, Duesseldorf, Germany.
⁷ Department of Otorhinolaryngology, Head and Neck Surgery, Paracelsus Medical University, Nuremberg, Germany.
⁸ Department of Otorhinolaryngology, Plastic Head and Neck Surgery, RWTH Aachen University Hospital, Aachen, Germany.
⁹ Department of Otorhinolaryngology, University Medical Center Freiburg, Freiburg, Germany.
¹⁰ Department of Otorhinolaryngology, Head and Neck Surgery, Technical University of Munich Hospital Rechts der Isar, Munich, Germany.
¹¹ Department of Radiology, LMU University Hospital, Munich, Germany.
¹² Ultrasound Division, Otorhinolaryngology Center Münsterland, Münster, Germany.
¹³ Rudolf-Frey Teaching Department, University Medical Center of the Johannes Gutenberg University Mainz, Mainz, Germany.
¹⁴ Department of Otorhinolaryngology, Universitätsklinikum Regensburg, Regensburg, Germany.

Abstract

Purpose Ultrasound examinations are often criticized for having higher examiner dependency compared to other imaging techniques. Compared to free-text reporting, structured reporting (SR) of head and neck sonography (HNS) achieves superior time efficiency as well as report quality. However, there are no findings concerning the influence of SR on the interrater reliability (IRR) of HNS. Materials and Methods Typical pathologies (n=4) in HNS were documented by video/images by two certified head and neck ultrasound instructors. Consequently, structured reports of these videos/images were created by n=9 senior physicians at departments of otolaryngology or maxillofacial surgery with DEGUM instructors on staff. Reports (n=36) were evaluated regarding overall completeness and IRR. Additionally, user satisfaction was assessed by a visual analog scale (VAS). Results SR yielded very high report completeness (91.8%) in all four cases with a substantial IRR (Fleiss' κ 0.73). Interrater agreement was high at 87.2% with very good user satisfaction (VAS 8.6). Conclusion SR has the potential to ensure high-quality examination reports with substantial comparability and very high user satisfaction. Furthermore, big data collection and analysis are facilitated by SR. Therefore, process quality, workflow, and scientific output are potentially enhanced by SR.

Keywords: interrater reliability; quality improvement; structured reporting; ultrasonography.

The Author(s). This is an open access article published by Thieme under the terms of the Creative Commons Attribution-NonDerivative-NonCommercial-License, permitting copying and reproduction so long as the original work is given appropriate credit. Contents may not be used for commercial purposes, or adapted, remixed, transformed or built upon. (https://creativecommons.org/licenses/by-nc-nd/4.0/).