Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

FunGraph: Functionality Aware 3D Scene Graphs for Language-Prompted Scene Interaction

Created by
  • Haebom

Author

Dennis Rotondi, Fabio Scaparro, Hermann Blum, Kai O. Arras

Outline

This paper aims to develop a 3D scene graph representation that identifies the location and usage of functional interaction elements, enabling robots to directly interact with their environments. Instead of relying on traditional object-level resolution, we focus on detecting and storing objects at a finer resolution, focusing on their functional significance. To address data scarcity and the challenges of capturing detailed object features using robotic sensors, we leverage existing 3D resources to generate 2D data and train detectors, enhancing the standard 3D scene graph generation pipeline. Experimental results demonstrate that our approach achieves functional element segmentation performance comparable to state-of-the-art 3D models and enables task-driven functional semantic associations with higher accuracy than existing solutions.

Takeaways, Limitations

Takeaways:
We present a novel 3D scene graph representation that identifies the location and usage of functional interaction elements.
Improving robot-environment interaction through fine-resolution object detection and storage.
Improving the performance of the 3D scene graph generation pipeline through 2D data augmentation.
Functional element segmentation and task-oriented functional semantic linking to achieve cutting-edge performance.
Limitations:
Difficulty in capturing detailed object features using robot sensors.
Lack of data beyond instance-level detection.
Utilization of limited range of 3D resources.
👍