UMRFormer-net: a three-dimensional U-shaped pancreas segmentation method based on a double-layer bridged transformer network

Kun Fang; Baochun He; Libo Liu; Haoyu Hu; Chihua Fang; Xuguang Huang; Fucang Jia

doi:10.21037/qims-22-544

UMRFormer-net: a three-dimensional U-shaped pancreas segmentation method based on a double-layer bridged transformer network

Quant Imaging Med Surg. 2023 Mar 1;13(3):1619-1630. doi: 10.21037/qims-22-544. Epub 2023 Feb 10.

Authors

Kun Fang^{1

2}, Baochun He², Libo Liu², Haoyu Hu³, Chihua Fang^{3

4}, Xuguang Huang¹, Fucang Jia^{2

4

5}

Affiliations

¹ School for Information and Optoelectronic Science and Engineering, South China Normal University, Guangzhou, China.
² Research Center for Medical Artificial Intelligence, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
³ Department of Hepatobiliary Surgery (I), Zhujiang Hospital of Southern Medical University, Guangzhou, China.
⁴ Pazhou Lab, Guangzhou, China.
⁵ Shenzhen Key Laboratory of Minimally Invasive Surgical Robotics and System, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.

Abstract

Background: Methods based on the combination of transformer and convolutional neural networks (CNNs) have achieved impressive results in the field of medical image segmentation. However, most of the recently proposed combination segmentation approaches simply treat transformers as auxiliary modules which help to extract long-range information and encode global context into convolutional representations, and there is a lack of investigation on how to optimally combine self-attention with convolution.

Methods: We designed a novel transformer block (MRFormer) that combines a multi-head self-attention layer and a residual depthwise convolutional block as the basic unit to deeply integrate both long-range and local spatial information. The MRFormer block was embedded between the encoder and decoder in U-Net at the last two layers. This framework (UMRFormer-Net) was applied to the segmentation of three-dimensional (3D) pancreas, and its ability to effectively capture the characteristic contextual information of the pancreas and surrounding tissues was investigated.

Results: Experimental results show that the proposed UMRFormer-Net achieved accuracy in pancreas segmentation that was comparable or superior to that of existing state-of-the-art 3D methods in both the Clinical Proteomic Tumor Analysis Consortium Pancreatic Ductal Adenocarcinoma (CPTAC-PDA) dataset and the public Medical Segmentation Decathlon dataset (self-division). UMRFormer-Net statistically significantly outperformed existing transformer-related methods and state-of-the-art 3D methods (P<0.05, P<0.01, or P<0.001), with a higher Dice coefficient (85.54% and 77.36%, respectively) or a lower 95% Hausdorff distance (4.05 and 8.34 mm, respectively).

Conclusions: UMRFormer-Net can obtain more matched and accurate segmentation boundary and region information in pancreas segmentation, thus improving the accuracy of pancreas segmentation. The code is available at https://github.com/supersunshinefk/UMRFormer-Net.

Keywords: Pancreas; U-Net; deep learning; image segmentation; transformer.