A brain-inspired theory of mind spiking neural network improves multi-agent cooperation and competition

Zhuoya Zhao; Feifei Zhao; Yuxuan Zhao; Yi Zeng; Yinqian Sun

doi:10.1016/j.patter.2023.100775

A brain-inspired theory of mind spiking neural network improves multi-agent cooperation and competition

Patterns (N Y). 2023 Jun 23;4(8):100775. doi: 10.1016/j.patter.2023.100775. eCollection 2023 Aug 11.

Authors

Zhuoya Zhao^{1

2}, Feifei Zhao¹, Yuxuan Zhao¹, Yi Zeng^{1

2

3

4

5}, Yinqian Sun^{1

2}

Affiliations

¹ Brain-inspired Cognitive Intelligence Lab, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.
² School of Future Technology, University of Chinese Academy of Sciences, Beijing 100049, China.
³ School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China.
⁴ Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai 200031, China.
⁵ State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China.

Abstract

During dynamic social interaction, inferring and predicting others' behaviors through theory of mind (ToM) is crucial for obtaining benefits in cooperative and competitive tasks. Current multi-agent reinforcement learning (MARL) methods primarily rely on agent observations to select behaviors, but they lack inspiration from ToM, which limits performance. In this article, we propose a multi-agent ToM decision-making (MAToM-DM) model, which consists of a MAToM spiking neural network (MAToM-SNN) module and a decision-making module. We design two brain-inspired ToM modules (Self-MAToM and Other-MAToM) to predict others' behaviors based on self-experience and observations of others, respectively. Each agent can adjust its behavior according to the predicted actions of others. The effectiveness of the proposed model has been demonstrated through experiments conducted in cooperative and competitive tasks. The results indicate that integrating the ToM mechanism can enhance cooperation and competition efficiency and lead to higher rewards compared with traditional MARL models.