Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learning

Created by
  • Haebom

Author

Zijian Guo, Ilker I\c{s}{\i}k, HM Sabbir Ahmad, Wenchao Li

Outline

This paper presents GenZ-LTL, a novel method utilizing Linear Temporal Logic (LTL) to solve the generalization problem in Reinforcement Learning (RL) for complex, temporally extended task goals and safety constraints. GenZ-LTL decomposes LTL specifications into a sequence of reach-avoid subgoals using the structure of Büchi automata. Unlike existing methods, it achieves zero-shot generalization by solving each subgoal one by one using a safe RL formulation, rather than conditioning on the subgoal sequence . Furthermore, it introduces a novel subgoal-induced observation reduction technique to mitigate the exponential complexity of subgoal-state combinations under realistic assumptions. Experimental results demonstrate that GenZ-LTL significantly outperforms existing methods in zero-shot generalization.

Takeaways, Limitations

Takeaways:
We present GenZ-LTL, a novel method that enables zero-shot generalization for LTL specifications.
Effectively handle complex LTL tasks through sub-goal decomposition based on Büchi automation.
Improving zero-shot generalization performance with a sequential sub-goal solving method using a safe RL formulation.
Alleviating complexity issues through sub-goal-induced observation reduction techniques.
Demonstrated superior zero-shot generalization performance compared to existing methods.
Limitations:
Lack of specific explanations of the realistic assumptions of the proposed sub-goal-induced observation reduction technique.
Additional experiments are needed to evaluate generalization performance for LTL specifications of various complexities.
Lack of application and presentation of experimental results for actual robotic systems.
Lack of detailed description of strategies for seeking alternatives when subgoals are impossible.
👍