iEnhancer-ELM: improve enhancer identification by extracting position-related multiscale contextual information based on enhancer language models

Jiahao Li; Zhourun Wu; Wenhao Lin; Jiawei Luo; Jun Zhang; Qingcai Chen; Junjie Chen

doi:10.1093/bioadv/vbad043

iEnhancer-ELM: improve enhancer identification by extracting position-related multiscale contextual information based on enhancer language models

Bioinform Adv. 2023 Mar 25;3(1):vbad043. doi: 10.1093/bioadv/vbad043. eCollection 2023.

Authors

Jiahao Li¹, Zhourun Wu¹, Wenhao Lin¹, Jiawei Luo¹, Jun Zhang¹, Qingcai Chen^{1

2}, Junjie Chen¹

Affiliations

¹ School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, Guangdong 518055, China.
² Guangdong Provincial Key Laboratory of Novel Security Intelligence Technologies, Harbin Institute of Technology, Shenzhen, Guangdong 518055, China.

Abstract

Motivation: Enhancers are important cis-regulatory elements that regulate a wide range of biological functions and enhance the transcription of target genes. Although many feature extraction methods have been proposed to improve the performance of enhancer identification, they cannot learn position-related multiscale contextual information from raw DNA sequences.

Results: In this article, we propose a novel enhancer identification method (iEnhancer-ELM) based on BERT-like enhancer language models. iEnhancer-ELM tokenizes DNA sequences with multi-scale k-mers and extracts contextual information of different scale k-mers related with their positions via an multi-head attention mechanism. We first evaluate the performance of different scale k-mers, then ensemble them to improve the performance of enhancer identification. The experimental results on two popular benchmark datasets show that our model outperforms state-of-the-art methods. We further illustrate the interpretability of iEnhancer-ELM. For a case study, we discover 30 enhancer motifs via a 3-mer-based model, where 12 of motifs are verified by STREME and JASPAR, demonstrating our model has a potential ability to unveil the biological mechanism of enhancer.

Availability and implementation: The models and associated code are available at https://github.com/chen-bioinfo/iEnhancer-ELM.

Supplementary information: Supplementary data are available at Bioinformatics Advances online.