Evaluating the Diagnostic Performance of Large Language Models on Complex Multimodal Medical Cases

Wan Hang Keith Chiu; Wei Sum Koel Ko; William Chi Shing Cho; Sin Yu Joanne Hui; Wing Chi Lawrence Chan; Michael D Kuo

doi:10.2196/53724

Evaluating the Diagnostic Performance of Large Language Models on Complex Multimodal Medical Cases

J Med Internet Res. 2024 May 13:26:e53724. doi: 10.2196/53724.

Authors

Wan Hang Keith Chiu¹, Wei Sum Koel Ko¹, William Chi Shing Cho², Sin Yu Joanne Hui³, Wing Chi Lawrence Chan⁴, Michael D Kuo⁵

Affiliations

¹ Department of Diagnostic and Interventional Radiology, Queen Elizabeth Hospital, Hong Kong, China (Hong Kong).
² Department of Clinical Oncology, Queen Elizabeth Hospital, Hong Kong, China (Hong Kong).
³ School of Biomedical Sciences, Li Ka Shing Faculty of Medicine, The University of Hong Kong, Hong Kong, China (Hong Kong).
⁴ Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong, China (Hong Kong).
⁵ Ensemble Group, Scottsdale, AZ, United States.

PMID: 38739441
DOI: 10.2196/53724

Abstract

Large language models showed interpretative reasoning in solving diagnostically challenging medical cases.

Keywords: ANOVA; Massachusetts; chi-square; clinician; disease etiology; health center; hospital; large language model; performance; physician; proficiency; statistical analysis.

©Wan Hang Keith Chiu, Wei Sum Koel Ko, William Chi Shing Cho, Sin Yu Joanne Hui, Wing Chi Lawrence Chan, Michael D Kuo. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 13.05.2024.

Publication types

Research Support, Non-U.S. Gov't

MeSH terms

Computer Simulation*
Diagnosis, Computer-Assisted*