ChatGPT's Gastrointestinal Tumor Board Tango: A limping dance partner?

Ughur Aghamaliyev; Javad Karimbayli; Clemens Giessen-Jung; Ilmer Matthias; Kristian Unger; Dorian Andrade; Felix O Hofmann; Maximilian Weniger; Martin K Angele; C Benedikt Westphalen; Jens Werner; Bernhard W Renz

doi:10.1016/j.ejca.2024.114100

ChatGPT's Gastrointestinal Tumor Board Tango: A limping dance partner?

Eur J Cancer. 2024 May 7:205:114100. doi: 10.1016/j.ejca.2024.114100. Online ahead of print.

Affiliations

¹ Department of General, Visceral and Transplantation Surgery, LMU University Hospital, LMU Munich, Germany.
² Division of Molecular Oncology, Centro di Riferimento Oncologico di Aviano (CRO), IRCCS, National Cancer Institute, Aviano, Italy.
³ Comprehensive Cancer Center Munich & Department of Medicine III, LMU University Hospital, LMU Munich, Germany.
⁴ Department of General, Visceral and Transplantation Surgery, LMU University Hospital, LMU Munich, Germany; German Cancer Consortium (DKTK), Partner Site Munich, Munich, Germany.
⁵ German Cancer Consortium (DKTK), Partner Site Munich, Munich, Germany; Department of Radiation Oncology, University Hospital, LMU Munich, 81377; Bavarian Cancer Research Center (BZKF), Munich, Germany.
⁶ Comprehensive Cancer Center Munich & Department of Medicine III, LMU University Hospital, LMU Munich, Germany; German Cancer Consortium (DKTK), Partner Site Munich, Munich, Germany.
⁷ Department of General, Visceral and Transplantation Surgery, LMU University Hospital, LMU Munich, Germany; German Cancer Consortium (DKTK), Partner Site Munich, Munich, Germany. Electronic address: bernhard.renz@med.uni-muenchen.de.

PMID: 38729055
DOI: 10.1016/j.ejca.2024.114100

Abstract

Objectives: This study aimed to assess the consistency and replicability of treatment recommendations provided by ChatGPT 3.5 compared to gastrointestinal tumor cases presented at multidisciplinary tumor boards (MTBs). It also aimed to distinguish between general and case-specific responses and investigated the precision of ChatGPT's recommendations in replicating exact treatment plans, particularly regarding chemotherapy regimens and follow-up protocols.

Material and methods: A retrospective study was carried out on 115 cases of gastrointestinal malignancies, selected from 448 patients reviewed in MTB meetings. A senior resident fed patient data into ChatGPT 3.5 to produce treatment recommendations, which were then evaluated against the tumor board's decisions by senior oncology fellows.

Results: Among the examined cases, ChatGPT 3.5 provided general information about the malignancy without considering individual patient characteristics in 19% of cases. However, only in 81% of cases, ChatGPT generated responses that were specific to the individual clinical scenarios. In the subset of case-specific responses, 83% of recommendations exhibited overall treatment strategy concordance between ChatGPT and MTB. However, the exact treatment concordance dropped to 65%, notably lower in recommending specific chemotherapy regimens. Cases recommended for surgery showed the highest concordance rates, while those involving chemotherapy recommendations faced challenges in precision.

Conclusions: ChatGPT 3.5 demonstrates potential in aligning conceptual approaches to treatment strategies with MTB guidelines. However, it falls short in accurately duplicating specific treatment plans, especially concerning chemotherapy regimens and follow-up procedures. Ethical concerns and challenges in achieving exact replication necessitate prudence when considering ChatGPT 3.5 for direct clinical decision-making in MTBs.

Keywords: AI; CCC; ChatGPT; Concordance; Gastrointestinal cancers; HCC; MTB; PDAC.