Feature Correlation-Steered Capsule Network for object detection

Zhongqi Lin; Jingdun Jia; Feng Huang; Wanlin Gao

doi:10.1016/j.neunet.2021.12.003

Feature Correlation-Steered Capsule Network for object detection

Neural Netw. 2022 Mar:147:25-41. doi: 10.1016/j.neunet.2021.12.003. Epub 2021 Dec 11.

Authors

Zhongqi Lin¹, Jingdun Jia², Feng Huang³, Wanlin Gao⁴

Affiliations

¹ College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China; Key Laboratory of Agricultural Informatization Standardization, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China.
² Key Laboratory of Agricultural Informatization Standardization, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China.
³ College of Science, China Agricultural University, Beijing, 100083, China. Electronic address: huangfeng@cau.edu.cn.
⁴ College of Information and Electrical Engineering, China Agricultural University, Beijing, 100083, China; Key Laboratory of Agricultural Informatization Standardization, Ministry of Agriculture and Rural Affairs, Beijing, 100083, China. Electronic address: gaowlin@cau.edu.cn.

PMID: 34953299
DOI: 10.1016/j.neunet.2021.12.003

Abstract

Despite Convolutional Neural Networks (CNNs) based approaches have been successful in objects detection, they predominantly focus on positioning discriminative regions while overlooking the internal holistic part-whole associations within objects. This would ultimately lead to the neglect of feature relationships between object and its parts as well as among those parts, both of which are significantly helpful for detecting discriminative parts. In this paper, we propose to "look insider the objects" by digging into part-whole feature correlations and take the attempts to leverage those correlations endowed by the Capsule Network (CapsNet) for robust object detection. Actually, highly correlated capsules across adjacent layers share high familiarity, which will be more likely to be routed together. In light of this, we take such correlations between different capsules of the preceding training samples as an awareness to constrain the subsequent candidate voting scope during the routing procedure, and a Feature Correlation-Steered CapsNet (FCS-CapsNet) with Locally-Constrained Expectation-Maximum (EM) Routing Agreement (LCEMRA) is proposed. Different from conventional EM routing, LCEMRA stipulates that only those relevant low-level capsules (parts) meeting the requirement of quantified intra-object cohesiveness can be clustered to make up high-level capsules (objects). In doing so, part-object associations can be dug by transformation weighting matrixes between capsules layers during such "part backtracking" procedure. LCEMRA enables low-level capsules to selectively gather projections from a non-spatially-fixed set of high-level capsules. Experiments on VOC2007, VOC2012, HKU-IS, DUTS, and COCO show that FCS-CapsNet can achieve promising object detection effects across multiple evaluation metrics, which are on-par with state-of-the-arts.

Keywords: Capsule Network (CapsNet); Expectation-maximum routing agreement; Feature correlation; Object detection; Part-object association.

MeSH terms

Neural Networks, Computer*