Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Episodic Memory Verbalization using Hierarchical Representations of Life-Long Robot Experience

Created by
  • Haebom

Author

Leonard B armann, Chad DeChant, Joana Plewnia, Fabian Peller-Konrad, Daniel Bauer, Tamim Asfour, Alex Waibel

Outline

This paper focuses on the ability of robots to summarize their long-term experiences and answer questions—that is, to verbalize robot experience. Unlike previous studies that apply rule-based systems or fine-tuned deep models to short-term experience data, which suffer from limited generalization and transferability, this study leverages a pre-trained, large-scale language model to verbalize long-term robot experience through zero- or few-shot learning. A hierarchical tree structure is derived from episodic memory (EM), representing raw sensory and proprioceptive data at lower levels and abstracted events as natural language concepts at higher levels. Based on user queries, the large-scale language model acts as an agent to interactively explore the EM, dynamically expanding tree nodes to find relevant information. This approach maintains low computational costs even with months of robot experience data. We evaluate the flexibility and scalability of our method using simulated domestic robot data, human-viewpoint video, and real-world robot recordings.

Takeaways, Limitations

•
Takeaways:
◦
A novel method for efficiently verbalizing robots' long-term experiences using large-scale language models is presented.
◦
Effectively manage and navigate EM through a hierarchical tree structure
◦
Improve generalization and transferability through zero-shot or few-shot learning.
◦
Validation of the method's flexibility and scalability through experiments on various datasets.
•
Limitations:
◦
Currently, only simulations and evaluations on limited real-world datasets are being conducted.
◦
Additional validation of performance and stability in real-world, complex environments is needed.
◦
Due to the nature of large language models, there may be explainability and reliability issues.
◦
The complexity of creating hierarchical tree structures and the need for optimization.
👍
Made with Slashpage