Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Episodic Memory Verbalization using Hierarchical Representations of Life-Long Robot Experience

Created by
  • Haebom

Author

Leonard B armann, Chad DeChant, Joana Plewnia, Fabian Peller-Konrad, Daniel Bauer, Tamim Asfour, Alex Waibel

Outline

This paper focuses on the ability of robots to summarize long-term experiences and answer questions—that is, to verbalize robot experience. While previous studies have limited generalization and transferability by applying rule-based systems or fine-tuned deep models to short-term experience data, this study leverages a pre-trained, large-scale language model to verbalize a robot's lifetime experience through zero- or few-shot learning. It generates hierarchical tree-structured data from episodic memories (EMs), representing raw sensory and proprioceptive data at low levels and abstracted events using natural language concepts at high levels. The large-scale language model acts as an agent to interactively explore the EMs based on user queries, dynamically expanding tree nodes to find relevant information. This approach maintains low computational costs even with months of robot experience data. We evaluate the flexibility and scalability of our method using simulated home robot data, human-viewpoint videos, and real-world robot recordings.

Takeaways, Limitations

Takeaways:
We present a novel method for efficiently verbalizing long-term experiences of robots by leveraging pre-trained large-scale language models.
Efficiently process large amounts of data and reduce computational costs through hierarchical tree structured data.
Performance validation on diverse datasets, including simulation data, human-viewpoint video, and real robot data.
Potential to contribute to improving human-robot interaction.
Limitations:
Currently, evaluations are limited to simulations and limited real-world data. Additional validation is needed on more diverse and complex real-world data.
Given the nature of large language models, further research is needed on explainability and reliability.
Further research is needed on optimization and generalization of tree structure design.
👍