Resilient multi-agent RL: introducing DQ-RTS for distributed environments with data loss

Lorenzo Canese; Gian Carlo Cardarilli; Luca Di Nunzio; Rocco Fazzolari; Marco Re; Sergio Spanò

doi:10.1038/s41598-023-48767-1

Resilient multi-agent RL: introducing DQ-RTS for distributed environments with data loss

Sci Rep. 2024 Jan 23;14(1):1994. doi: 10.1038/s41598-023-48767-1.

Authors

Lorenzo Canese^#¹, Gian Carlo Cardarilli^#², Luca Di Nunzio^#², Rocco Fazzolari^#², Marco Re^#², Sergio Spanò^#²

Affiliations

¹ Department of Electronics, University of Rome Tor Vergata, 00133, Rome, Italy. canese@ing.uniroma2.it.
² Department of Electronics, University of Rome Tor Vergata, 00133, Rome, Italy.

^# Contributed equally.

Abstract

This paper proposes DQ-RTS, a novel decentralized Multi-Agent Reinforcement Learning algorithm designed to address challenges posed by non-ideal communication and a varying number of agents in distributed environments. DQ-RTS incorporates an optimized communication protocol to mitigate data loss between agents. A comparative analysis between DQ-RTS and its decentralized counterpart Q-RTS, or Q-learning for Real-Time Swarms, demonstrates the superior convergence speed of DQ-RTS, achieving a remarkable speed-up factor ranging from 1.6 to 2.7 in scenarios with non-ideal communication. Moreover, DQ-RTS exhibits robustness by maintaining performance even when the agent population fluctuates, making it well-suited for applications requiring adaptable agent numbers over time. Additionally, extensive experiments conducted on various benchmark tasks validate the scalability and effectiveness of DQ-RTS, further establishing its potential as a practical solution for resilient Multi-Agent Reinforcement Learning in dynamic distributed environments.