Background: Generative artificial intelligence (AI) models offer potential assistance in pain research data acquisition, yet concerns persist regarding data accuracy and reliability. In a comparative study, we evaluated open generative AI models' capacity to acquire data on acute pain in rock climbers comparable to field research.
Methods: Fifty-two rock climbers (33 m/19 f; age 29.0 [24.0-35.75] years) were asked to report pain location and intensity during a single climbing session. Five generative pretrained transformer models were tasked with responses to the same questions.
Results: Climbers identified the back of the forearm (19.2%) and toes (17.3%) as primary pain sites, with reported median pain intensity at 4 [3-5] and median maximum pain intensity at 7 [5-8]. Conversely, AI models yielded divergent findings, indicating fingers, hands, shoulders, legs, and feet as primary pain localizations with average and maximum pain intensity ranging from 3 to 4.4 and 5 to 10, respectively. Only two AI models provided references that were untraceable in PubMed and Google searches.
Conclusion: Our findings reveal that, currently, open generative AI models cannot match the quality of field-collected data on acute pain in rock climbers. Moreover, the models generated nonexistent references, raising concerns about their reliability.
Keywords: acute pain; artificial intelligence; climbing; generative pretrained transformer.
© 2024 the author(s), published by De Gruyter.