Background: Artificial intelligence is increasingly being used in all aspects of life in information compilation and writing, and this includes healthcare. This study aimed to evaluate a Chat GPT generated patient information leaflet (PIL) against a surgeon generated version, in order to explore a potential application of this artificial intelligence language processing model.
Methods: Cross-sectional study, undertaken May to June 2023, asking two cohorts (patients and doctors) to complete a questionnaire evaluating a Chat GPT generated PIL and a surgeon generated PIL about laparoscopic cholecystectomy. The patients were having laparoscopic cholecystectomy at large private Hospital in Melbourne, Australia, and doctors were recruited from this hospital and a public quaternary hospital in Melbourne, Australia. The study included a convenience sample of 28 patients and 16 doctors. The main outcome measure was a questionnaire (maximum score out of 8) based on validated evaluation instrument for PILs.
Results: The study recruited 28 patients and 15 doctors to complete the questionnaire. The Chat GPT and surgeon generated PILs were scored similarly by patients (median 8 for both PIL; mean 7.5 for Chat GPT PIL vs. 7.1 for surgeon PIL). Doctors also scored both versions similarly, with slightly higher scores for Chat GPT over surgeon version (median 7 vs. 6; mean 6.7 vs. 5.6, respectively).
Conclusions: The Chat GPT generated PIL was assessed as being as good or slightly better than the surgeon generated version. This study shows that PIL are a feasible application of AI language processing models.
Keywords: artificial intelligence; general surgery; patient information; surgical education.
© 2023 Royal Australasian College of Surgeons.