Survey on natural language processing in medical image analysis
Zhong Nan Da Xue Xue Bao Yi Xue Ban. 2022 Aug 28;47(8):981-993.
doi: 10.11817/j.issn.1672-7347.2022.220376.
[Article in
English,
Chinese]
Authors
Zhengliang Liu
1
, Mengshen He
2
, Zuowei Jiang
3
, Zihao Wu
4
, Haixing Dai
4
, Lian Zhang
5
, Siyi Luo
6
, Tianle Han
2
, Xiang Li
7
, Xi Jiang
8
, Dajiang Zhu
9
, Xiaoyan Cai
3
, Bao Ge
2
, Wei Liu
5
, Jun Liu
10
, Dinggang Shen
11
, Tianming Liu
4
Affiliations
- 1 Department of Computer Science, University of Georgia, Athens, GA 30602, USA. zl18864@uga.edu.
- 2 School of Physics & Information Technology, Shaanxi Normal University, Xi'an 710119, China.
- 3 School of Automation, Northwestern Polytechnical University, Xi'an 710072, China.
- 4 Department of Computer Science, University of Georgia, Athens, GA 30602, USA.
- 5 Department of Radiation Oncology, Mayo Clinic, Phoenix, AZ 85054, USA.
- 6 Faculty of Electronic and Information Engineering, Xi'an Jiaotong University, Xi'an 710049, China.
- 7 Massachusetts General Hospital and Harvard Medical School, Boston, MA 02114, USA.
- 8 School of Life Science and Technology, University of Electronic Science and Technology, Chengdu 611731, China.
- 9 Department of Computer Science and Engineering, University of Texas at Arlington, Arlington, TX 76019, USA.
- 10 Department of Radiology, Second Xiangya Hospital, Central South University, Changsha 410011, China.
- 11 School of Biomedical Engineering, ShanghaiTech University, Shanghai 201210, China.
Recent advancement in natural language processing (NLP) and medical imaging empowers the wide applicability of deep learning models. These developments have increased not only data understanding, but also knowledge of state-of-the-art architectures and their real-world potentials. Medical imaging researchers have recognized the limitations of only targeting images, as well as the importance of integrating multimodal inputs into medical image analysis. The lack of comprehensive surveys of the current literature, however, impedes the progress of this domain. Existing research perspectives, as well as the architectures, tasks, datasets, and performance measures examined in the present literature, are reviewed in this work, and we also provide a brief description of possible future directions in the field, aiming to provide researchers and healthcare professionals with a detailed summary of existing academic research and to provide rational insights to facilitate future research.
自然语言处理和医学影像的发展使深度学习模型能够在各种领域和各种数据模态中表现出出色的通用性。这些进步不仅加深了对数据的理解，而且促进了学界对最先进架构及其前景的认识。医学影像研究人员已经认识到仅针对图像研究的不足之处，以及对多模态输入进行综合分析的重要意义。但是，目前相关综述论文的缺乏不利于这个研究方向的发展。本篇综述介绍了自然语言处理和医学图像结合这一领域的背景，并分5个主题回顾现有文献的研究目标、模型架构、目标任务、实验数据和性能指标，还对该领域未来可能的发展方向进行简要描述，旨在为研究人员和医护人员提供现有学术研究的详细总结，提出理性的见解，进而促进未来的研究。.
Keywords:
deep learning; medical imaging; multimodal input; natural language processing.
MeSH terms
-
Humans
-
Natural Language Processing*
-
Surveys and Questionnaires