Chat GPT for the management of obstructive sleep apnea: do we have a polar star?

Felipe Ahumada Mira; Valentin Favier; Heloisa Dos Santos Sobreira Nunes; Joana Vaz de Castro; Florent Carsuzaa; Giuseppe Meccariello; Claudio Vicini; Andrea De Vito; Jerome R Lechien; Carlos Chiesa-Estomba; Antonino Maniaci; Giannicola Iannella; Eduardo Peña Rojas; Jenifer Barros Cornejo; Giovanni Cammaroto

doi:10.1007/s00405-023-08270-9

Chat GPT for the management of obstructive sleep apnea: do we have a polar star?

Eur Arch Otorhinolaryngol. 2024 Apr;281(4):2087-2093. doi: 10.1007/s00405-023-08270-9. Epub 2023 Nov 19.

Authors

Felipe Ahumada Mira^{1

2}, Valentin Favier^{3

2}, Heloisa Dos Santos Sobreira Nunes^{4

2}, Joana Vaz de Castro^{5

2}, Florent Carsuzaa^{6

2}, Giuseppe Meccariello⁷, Claudio Vicini⁷, Andrea De Vito⁷, Jerome R Lechien^{8

2}, Carlos Chiesa-Estomba^{9

2}, Antonino Maniaci^{10

2}, Giannicola Iannella^{11

2}, Eduardo Peña Rojas¹², Jenifer Barros Cornejo¹³, Giovanni Cammaroto^{14

15}

Affiliations

¹ ENT Department, Hospital of Linares, Linares, Chile.
² Young Otolaryngologists-International Federations of Oto-Rhinolaryngological Societies (YO-IFOS), Paris, France.
³ ENT Department, University Hospital of Montpellier, Montpellier, France.
⁴ ENT and Sleep Medicine Department, Nucleus of Otolaryngology, Head and Neck Surgery and Sleep Medicine of São Paulo, São Paulo, Brazil.
⁵ ENT Department, Armed Forces Hospital, Lisbon, Portugal.
⁶ ENT Department, University Hospital of Poitiers, Poitiers, France.
⁷ Head and Neck Department, ENT & Oral Surgery Unity, G.B. Morgagni, L. Pierantoni Hospital, Via Forlanini, 47121, Forlì, Italy.
⁸ Division of Laryngology and Broncho-Esophagology, Department of Otolaryngology and Head and Neck Surgery, EpiCURA Hospital, UMONS Research Institute for Health Sciences and Technology, University of Mons, Mons, Belgium.
⁹ Department of Otorhinolaryngology, Biodonostia Research Institute, Donostia University Hospital, Osakidetza, 20014, San Sebastian, Spain.
¹⁰ Department of Medical and Surgical Sciences and Advanced Technologies "GF Ingrassia", ENT Section, University of Catania, Piazza Università 2, 95100, Catania, Italy.
¹¹ Department of 'Organi di Senso', University "Sapienza", Viale Dell'Università 33, 00185, Rome, Italy.
¹² Clínica Lircay, Talca, Chile.
¹³ Hospital Clínico UC Christus, Santiago, Chile.
¹⁴ Head and Neck Department, ENT & Oral Surgery Unity, G.B. Morgagni, L. Pierantoni Hospital, Via Forlanini, 47121, Forlì, Italy. giovanni.cammaroto@hotmail.com.
¹⁵ Young Otolaryngologists-International Federations of Oto-Rhinolaryngological Societies (YO-IFOS), Paris, France. giovanni.cammaroto@hotmail.com.

PMID: 37980605
DOI: 10.1007/s00405-023-08270-9

Abstract

Purpose: This study explores the potential of the Chat-Generative Pre-Trained Transformer (Chat-GPT), a Large Language Model (LLM), in assisting healthcare professionals in the diagnosis of obstructive sleep apnea (OSA). It aims to assess the agreement between Chat-GPT's responses and those of expert otolaryngologists, shedding light on the role of AI-generated content in medical decision-making.

Methods: A prospective, cross-sectional study was conducted, involving 350 otolaryngologists from 25 countries who responded to a specialized OSA survey. Chat-GPT was tasked with providing answers to the same survey questions. Responses were assessed by both super-experts and statistically analyzed for agreement.

Results: The study revealed that Chat-GPT and expert responses shared a common answer in over 75% of cases for individual questions. However, the overall consensus was achieved in only four questions. Super-expert assessments showed a moderate agreement level, with Chat-GPT scoring slightly lower than experts. Statistically, Chat-GPT's responses differed significantly from experts' opinions (p = 0.0009). Sub-analysis revealed areas of improvement for Chat-GPT, particularly in questions where super-experts rated its responses lower than expert consensus.

Conclusions: Chat-GPT demonstrates potential as a valuable resource for OSA diagnosis, especially where access to specialists is limited. The study emphasizes the importance of AI-human collaboration, with Chat-GPT serving as a complementary tool rather than a replacement for medical professionals. This research contributes to the discourse in otolaryngology and encourages further exploration of AI-driven healthcare applications. While Chat-GPT exhibits a commendable level of consensus with expert responses, ongoing refinements in AI-based healthcare tools hold significant promise for the future of medicine, addressing the underdiagnosis and undertreatment of OSA and improving patient outcomes.

Keywords: Chat-Gpt; Chatbot; OSA; Sleep apnea.

MeSH terms

Alanine Transaminase
Clinical Decision-Making*
Cross-Sectional Studies
Humans
Prospective Studies
Sleep Apnea, Obstructive* / diagnosis
Sleep Apnea, Obstructive* / therapy

Substances

Alanine Transaminase