ESCaPe 2020 Proceedings | Page 12

Multi-Agent Reinforcement Learning in Sparsely Connected Cooperative Environments Pankayaraj Pathmanathan 1 , Chandima Samarasinghe 1 , Yuvini Sumanasekera 1* , Dhammika Elkaduwe 1 , Upul Jayasinghe 1 and D. H. S. Maithripala 2 1 Department of Computer Engineering, Faculty of Engineering, University of Peradeniya, Sri Lanka 2 Department of Mechanical Engineering, Faculty of Engineering, University of Peradeniya, Sri Lanka *E-mail: [email protected] Abstract: In multi-agent systems, an agent’s behavior is affected by the dynamicity of the environment as well as by the interactions among agents. Thus, in learning to cooperate to solve complex tasks, such considerations should be taken into account. Reinforcement learning has gained immense interest in this line of research as it allows agents to learn useful behavior by dynamically interacting with the environment and with one another. In this work, we exploit the inherent graph-like structure of multi-agent networks to facilitate the learning of more robust behavior strategies by capturing the spatial dependencies and temporal dynamics of the underlying graph. However, partial observability, as well as restricted communication can result in agents learning suboptimal strategies. We address these issues by allowing each agent to recurrently propagate information through its neighborhood, thus gradually increasing its receptive field. Finally, we demonstrate the effectiveness of the proposed model on a variety of cooperative control tasks. Key Words - multi-agent, reinforcement learning, spatio-temporal dependencies, graph neural networks 12