Are artificial intelligence chatbots a reliable source of information about contact lenses?

Abstract

Introduction: Artificial Intelligence (AI) chatbots are able to explain complex concepts using plain language. The aim of this study was to assess the accuracy of three AI chatbots answering common questions related to contact lens (CL) wear.

Methods: Three open access AI chatbots were compared: Perplexity, Open Assistant and ChatGPT 3.5. Ten general CL questions were asked to all AI chatbots on the same day in two different countries, with the questions asked in Spanish from Spain and in English from the U.K. Two independent optometrists with experience working in each country assessed the accuracy of the answers provided. Also, the AI chatbots' responses were assessed if their outputs showed any bias towards (or against) any eye care professional (ECP).

Results: The answers obtained by the same AI chatbots were different in Spain and the U.K. Also, statistically significant differences were found between the AI chatbots for accuracy. In the U.K., ChatGPT 3.5 was the most and Open Assistant least accurate (p < 0.01). In Spain, Perplexity and ChatGPT were statistically more accurate than Open Assistant (p < 0.01). All the AI chatbots presented bias, except ChatGPT 3.5 in Spain.

Conclusions: AI chatbots do not always consider local CL legislation, and their accuracy seems to be dependent on the language used to interact with them. Hence, at this time, although some AI chatbots might be a good source of information for general CL related questions, they cannot replace an ECP.

Keywords: Artificial intelligence chatbots; Contact lens opticians; Contact lenses; Eye care professionals; Ophthalmologists; Optometrists.