Attention based automated radiology report generation using CNN and LSTM

PLoS One. 2022 Jan 6;17(1):e0262209. doi: 10.1371/journal.pone.0262209. eCollection 2022.

Abstract

The automated generation of radiology reports provides X-rays and has tremendous potential to enhance the clinical diagnosis of diseases in patients. A new research direction is gaining increasing attention that involves the use of hybrid approaches based on natural language processing and computer vision techniques to create auto medical report generation systems. The auto report generator, producing radiology reports, will significantly reduce the burden on doctors and assist them in writing manual reports. Because the sensitivity of chest X-ray (CXR) findings provided by existing techniques not adequately accurate, producing comprehensive explanations for medical photographs remains a difficult task. A novel approach to address this issue was proposed, based on the continuous integration of convolutional neural networks and long short-term memory for detecting diseases, followed by the attention mechanism for sequence generation based on these diseases. Experimental results obtained by using the Indiana University CXR and MIMIC-CXR datasets showed that the proposed model attained the current state-of-the-art efficiency as opposed to other solutions of the baseline. BLEU-1, BLEU-2, BLEU-3, and BLEU-4 were used as the evaluation metrics.

MeSH terms

  • Algorithms*
  • Deep Learning*
  • Humans
  • Natural Language Processing*
  • Neural Networks, Computer*
  • Radiography, Thoracic / methods*
  • Radiology*
  • X-Rays

Grants and funding

This research project was funded by the Deanship of Scientific Research, Princess Nourah bint Abdulrahman University, through the “Program of Research Project Funding After Publication, grant No (42-PRFA-P-53)”. https://www.pnu.edu.sa/en/Pages/home.aspx. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.