Performance of Generative Artificial Intelligence in Dental Licensing Examinations

Reinhard Chun Wang Chau; Khaing Myat Thu; Ollie Yiru Yu; Richard Tai-Chiu Hsung; Edward Chin Man Lo; Walter Yu Hang Lam

doi:10.1016/j.identj.2023.12.007

Performance of Generative Artificial Intelligence in Dental Licensing Examinations

Int Dent J. 2024 Jan 18:S0020-6539(23)00989-9. doi: 10.1016/j.identj.2023.12.007. Online ahead of print.

Authors

Reinhard Chun Wang Chau¹, Khaing Myat Thu¹, Ollie Yiru Yu¹, Richard Tai-Chiu Hsung², Edward Chin Man Lo¹, Walter Yu Hang Lam³

Affiliations

¹ Faculty of Dentistry, The University of Hong Kong, Hong Kong Special Administrative Region, China.
² Faculty of Dentistry, The University of Hong Kong, Hong Kong Special Administrative Region, China; Department of Computer Science, Hong Kong Chu Hai College, Hong Kong Special Administrative Region, China.
³ Faculty of Dentistry, The University of Hong Kong, Hong Kong Special Administrative Region, China; Musketeers Foundation Institute of Data Science, The University of Hong Kong, Hong Kong Special Administrative Region, China. Electronic address: retlaw@hku.hk.

PMID: 38242810
DOI: 10.1016/j.identj.2023.12.007

Abstract

Objectives: Generative artificial intelligence (GenAI), including large language models (LLMs), has vast potential applications in health care and education. However, it is unclear how proficient LLMs are in interpreting written input and providing accurate answers in dentistry. This study aims to investigate the accuracy of GenAI in answering questions from dental licensing examinations.

Methods: A total of 1461 multiple-choice questions from question books for the US and the UK dental licensing examinations were input into 2 versions of ChatGPT 3.5 and 4.0. The passing rates of the US and UK dental examinations were 75.0% and 50.0%, respectively. The performance of the 2 versions of GenAI in individual examinations and dental subjects was analysed and compared.

Results: ChatGPT 3.5 correctly answered 68.3% (n = 509) and 43.3% (n = 296) of questions from the US and UK dental licensing examinations, respectively. The scores for ChatGPT 4.0 were 80.7% (n = 601) and 62.7% (n = 429), respectively. ChatGPT 4.0 passed both written dental licensing examinations, whilst ChatGPT 3.5 failed. ChatGPT 4.0 answered 327 more questions correctly and 102 incorrectly compared to ChatGPT 3.5 when comparing the 2 versions.

Conclusions: The newer version of GenAI has shown good proficiency in answering multiple-choice questions from dental licensing examinations. Whilst the more recent version of GenAI generally performed better, this observation may not hold true in all scenarios, and further improvements are necessary. The use of GenAI in dentistry will have significant implications for dentist-patient communication and the training of dental professionals.

Keywords: Artificial intelligence; Communication; Dental education; Digital technology; Examination questions; Specialties, Dental.