Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

RIZE: Regularized Imitation Learning via Distributional Reinforcement Learning

Created by
  • Haebom

Author

Adib Karimi, Mohammad Mehdi Ebadzadeh

Outline

This paper proposes a novel inverse reinforcement learning (IRL) method that addresses the rigidity of fixed reward structures and the inflexibility of implicit reward regulation. Based on the maximum entropy IRL framework, it incorporates a squared temporal difference (TD) regularizer with an adaptive target that dynamically evolves during training, imposing adaptive bounds on restored rewards and facilitating robust decision-making. To capture richer payoff information, distributional reinforcement learning is incorporated into the training process. Experimentally, the proposed method achieves expert-level performance on the complex MuJoCo task and outperforms baseline methods on humanoid tasks across three demonstrations. Extensive experiments and ablation studies further validate the effectiveness of this method and provide insights into reward dynamics in imitation learning.

Takeaways, Limitations

Takeaways:
A novel inverse reinforcement learning method that overcomes the limitations of fixed reward structures is presented.
Promoting robust decision-making through a squared-time regularizer with adaptive targets.
Leverage richer revenue information through distributed reinforcement learning integration.
Achieve expert-level performance and surpass benchmark methods on complex MuJoCo tasks.
Provides new insights into reward dynamics
Limitations:
Further research is needed on the generalization performance of the proposed method.
Applicability verification for various environments and tasks is required.
The need to address computational cost issues in high-dimensional state spaces
👍