Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Symmetry-Guided Multi-Agent Inverse Reinforcement Learning

Created by
  • Haebom

Author

Yongkai Tian, Yirong Qi, Xin Yu, Wenjun Wu, Jie Luo

Outline

This paper addresses the problem that the performance of reinforcement learning in robotic systems relies on the rationality of predefined reward functions, but manually designed reward functions can lead to policy failure due to inaccuracies. Inverse reinforcement learning (IRL) addresses this problem by inferring implicit reward functions from expert demonstrations, but existing methods rely heavily on a large number of expert demonstrations to recover accurate reward functions. The high cost of collecting expert demonstrations, especially in multi-robot systems, severely hinders the practical deployment of IRL. Therefore, improving sampling efficiency has emerged as a critical challenge in multi-agent inverse reinforcement learning (MIRL). This paper theoretically demonstrates that leveraging the inherent symmetry in multi-agent systems can recover more accurate reward functions. Based on this insight, we propose a general framework that incorporates symmetry into existing multi-agent adversarial IRL algorithms, significantly improving sampling efficiency. Experimental results on various challenging tasks demonstrate the effectiveness of this framework, and further validation on real-world multi-robot systems demonstrates the practicality of our method.

Takeaways, Limitations

Takeaways:
We present a novel framework that significantly improves the sampling efficiency of MIRL by exploiting the symmetry of multi-agent systems.
The effectiveness of the proposed framework is experimentally verified through various complex tasks.
Verifying practicality in real multi-robot systems.
Limitations:
The possibility that the performance of the proposed framework depends on a specific type of symmetry.
Further research is needed on generalizability to various multi-agent systems.
Further research is needed on robustness to noise and uncertainty in real-world environments.
👍