Do You Need Embeddings Trained on a Massive Specialized Corpus for Your Clinical Natural Language Processing Task?

Antoine Neuraz; Vincent Looten; Bastien Rance; Nicolas Daniel; Nicolas Garcelon; Leonardo Campillos Llanos; Anita Burgun; Sophie Rosset

doi:10.3233/SHTI190533

Do You Need Embeddings Trained on a Massive Specialized Corpus for Your Clinical Natural Language Processing Task?

Stud Health Technol Inform. 2019 Aug 21:264:1558-1559. doi: 10.3233/SHTI190533.

Authors

Antoine Neuraz^{1

2

3}, Vincent Looten^{2

4}, Bastien Rance^{2

4}, Nicolas Daniel⁴, Nicolas Garcelon^{1

5}, Leonardo Campillos Llanos³, Anita Burgun^{1

2

4}, Sophie Rosset³

Affiliations

¹ Institut National de la Santé et de la Recherche Médicale (INSERM), Centre de Recherche des Cordeliers, UMR 1138 Equipe 22, Paris Descartes, Sorbonne Paris Cité University, Paris, France.
² Department of Medical Informatics, Necker-Enfants Malades Hospital, Assistance Publique des Hôpitaux de Paris (AP-HP).
³ LIMSI, CNRS, Université Paris Saclay.
⁴ Hôpital Européen Georges Pompidou, AP-HP, Université Paris Descartes, Sorbonne Paris Cité, Paris, France.
⁵ Institut Imagine, Paris Descartes Université Paris Descartes-Sorbonne Paris Cité, Paris, France.

PMID: 31438230
DOI: 10.3233/SHTI190533

Abstract

We explore the impact of data source on word representations for different NLP tasks in the clinical domain in French (natural language understanding and text classification). We compared word embeddings (Fasttext) and language models (ELMo), learned either on the general domain (Wikipedia) or on specialized data (electronic health records, EHR). The best results were obtained with ELMo representations learned on EHR data for one of the two tasks(+7% and +8% of gain in F1-score).

Keywords: Natural language processing; electronic health records.

MeSH terms

Electronic Health Records*
Histological Techniques
Language
Natural Language Processing*