MEDUSA: Multi-Scale Encoder-Decoder Self-Attention Deep Neural Network Architecture for Medical Image Analysis

Hossein Aboutalebi; Maya Pavlova; Hayden Gunraj; Mohammad Javad Shafiee; Ali Sabri; Amer Alaref; Alexander Wong

doi:10.3389/fmed.2021.821120

MEDUSA: Multi-Scale Encoder-Decoder Self-Attention Deep Neural Network Architecture for Medical Image Analysis

Front Med (Lausanne). 2022 Feb 15:8:821120. doi: 10.3389/fmed.2021.821120. eCollection 2021.

Authors

Hossein Aboutalebi^{1

2}, Maya Pavlova³, Hayden Gunraj³, Mohammad Javad Shafiee^{1

2

3}, Ali Sabri⁴, Amer Alaref^{5

6}, Alexander Wong^{1

2

3}

Affiliations

¹ Department of Computer Science, University of Waterloo, Waterloo, ON, Canada.
² Waterloo AI Institute, University of Waterloo, Waterloo, ON, Canada.
³ Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada.
⁴ Department of Radiology, Niagara Health, McMaster University, Hamilton, ON, Canada.
⁵ Department of Diagnostic Imaging, Northern Ontario School of Medicine, Thunder Bay, ON, Canada.
⁶ Department of Diagnostic Radiology, Thunder Bay Regional Health Sciences Centre, Thunder Bay, ON, Canada.

Abstract

Medical image analysis continues to hold interesting challenges given the subtle characteristics of certain diseases and the significant overlap in appearance between diseases. In this study, we explore the concept of self-attention for tackling such subtleties in and between diseases. To this end, we introduce, a multi-scale encoder-decoder self-attention (MEDUSA) mechanism tailored for medical image analysis. While self-attention deep convolutional neural network architectures in existing literature center around the notion of multiple isolated lightweight attention mechanisms with limited individual capacities being incorporated at different points in the network architecture, MEDUSA takes a significant departure from this notion by possessing a single, unified self-attention mechanism with significantly higher capacity with multiple attention heads feeding into different scales in the network architecture. To the best of the authors' knowledge, this is the first "single body, multi-scale heads" realization of self-attention and enables explicit global context among selective attention at different levels of representational abstractions while still enabling differing local attention context at individual levels of abstractions. With MEDUSA, we obtain state-of-the-art performance on multiple challenging medical image analysis benchmarks including COVIDx, Radiological Society of North America (RSNA) RICORD, and RSNA Pneumonia Challenge when compared to previous work. Our MEDUSA model is publicly available.

Keywords: COVID-19; chest X-ray (CXR); computer vision; deep neural net; diagnosis.