Multi-Agent Reinforcement Learning in
Sparsely Connected Cooperative Environments
Pankayaraj Pathmanathan 1 , Chandima Samarasinghe 1 , Yuvini Sumanasekera 1* , Dhammika Elkaduwe 1 , Upul
Jayasinghe 1 and D. H. S. Maithripala 2
1
Department of Computer Engineering, Faculty of Engineering, University of Peradeniya, Sri Lanka
2
Department of Mechanical Engineering, Faculty of Engineering, University of Peradeniya, Sri Lanka
*E-mail: [email protected]
Abstract: In multi-agent systems, an agent’s behavior is affected by the dynamicity of the environment as well
as by the interactions among agents. Thus, in learning to cooperate to solve complex tasks, such considerations
should be taken into account. Reinforcement learning has gained immense interest in this line of research as it
allows agents to learn useful behavior by dynamically interacting with the environment and with one another.
In this work, we exploit the inherent graph-like structure of multi-agent networks to facilitate the learning of
more robust behavior strategies by capturing the spatial dependencies and temporal dynamics of the underlying
graph. However, partial observability, as well as restricted communication can result in agents learning suboptimal
strategies. We address these issues by allowing each agent to recurrently propagate information through
its neighborhood, thus gradually increasing its receptive field. Finally, we demonstrate the effectiveness of the
proposed model on a variety of cooperative control tasks.
Key Words - multi-agent, reinforcement learning, spatio-temporal dependencies, graph neural networks
12