Performance evaluation of Unified Medical Language System®'s synonyms expansion to query PubMed

BMC Med Inform Decis Mak. 2012 Feb 29:12:12. doi: 10.1186/1472-6947-12-12.

Abstract

Background: PubMed is the main access to medical literature on the Internet. In order to enhance the performance of its information retrieval tools, primarily non-indexed citations, the authors propose a method: expanding users' queries using Unified Medical Language System' (UMLS) synonyms i.e. all the terms gathered under one unique Concept Unique Identifier.

Methods: This method was evaluated using queries constructed to emphasize the differences between this new method and the current PubMed automatic term mapping. Four experts assessed citation relevance.

Results: Using UMLS, we were able to retrieve new citations in 45.5% of queries, which implies a small increase in recall. The new strategy led to a heterogeneous 23.7% mean increase in non-indexed citation retrieved. Of these, 82% have been published less than 4 months earlier. The overall mean precision was 48.4% but differed according to the evaluators, ranging from 36.7% to 88.1% (Inter rater agreement was poor: kappa = 0.34).

Conclusions: This study highlights the need for specific search tools for each type of user and use-cases. The proposed strategy may be useful to retrieve recent scientific advancement.

Publication types

  • Evaluation Study

MeSH terms

  • Information Storage and Retrieval / methods*
  • Medical Subject Headings*
  • PubMed*
  • Reproducibility of Results
  • Unified Medical Language System / standards*
  • User-Computer Interface