EHR2Vec: Representation Learning of Medical Concepts From Temporal Patterns of Clinical Notes Based on Self-Attention Mechanism

Li Wang; Qinghua Wang; Heming Bai; Cong Liu; Wei Liu; Yuanpeng Zhang; Lei Jiang; Huji Xu; Kai Wang; Yunyun Zhou

doi:10.3389/fgene.2020.00630

EHR2Vec: Representation Learning of Medical Concepts From Temporal Patterns of Clinical Notes Based on Self-Attention Mechanism

Front Genet. 2020 Jun 29:11:630. doi: 10.3389/fgene.2020.00630. eCollection 2020.

Authors

Li Wang^{1

2}, Qinghua Wang¹, Heming Bai², Cong Liu³, Wei Liu², Yuanpeng Zhang^{1

2}, Lei Jiang⁴, Huji Xu^{4

5

6}, Kai Wang^{7

8}, Yunyun Zhou⁷

Affiliations

¹ Department of Medical Informatics, Medical School, Nantong University, Nantong, China.
² Research Center for Intelligence Information Technology, Nantong University, Nantong, China.
³ Department of Biomedical Informatics, Columbia University, New York, NY, United States.
⁴ Department of Rheumatology and Immunology, Changzheng Hospital, The Second Military Medical University, Shanghai, China.
⁵ Beijing Tsinghua Chang Gung Hospital, School of Clinical Medicine, Tsinghua University, Beijing, China.
⁶ Peking-Tsinghua Center for Life Sciences, Tsinghua University, Beijing, China.
⁷ Raymond G. Perelman Center for Cellular and Molecular Therapeutics, Children's Hospital of Philadelphia, Philadelphia, PA, United States.
⁸ Department of Pathology and Laboratory Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States.

Abstract

Efficiently learning representations of clinical concepts (i. e., symptoms, lab test, etc.) from unstructured clinical notes of electronic health record (EHR) data remain significant challenges, since each patient may have multiple visits at different times and each visit may contain different sequential concepts. Therefore, learning distributed representations from temporal patterns of clinical notes is an essential step for downstream applications on EHR data. However, existing methods for EHR representation learning can not adequately capture either contextual information per-visit or temporal information at multiple visits. In this study, we developed a new vector embedding method called EHR2Vec that can learn semantically-meaningful representations of clinical concepts. EHR2Vec incorporated the self-attention structure and showed its utility in accurately identifying relevant clinical concept entities considering time sequence information from multiple visits. Using EHR data from systemic lupus erythematosus (SLE) patients as a case study, we showed EHR2Vec outperforms in identifying interpretable representations compared to other well-known methods including Word2Vec and Med2Vec, according to clinical experts' evaluations.

Keywords: electronic health record; natural language processing; representation learning; unstructured clinical notes; word vector.