Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Real-is-Sim: Bridging the Sim-to-Real Gap with a Dynamic Digital Twin

Created by
  • Haebom

Author

Jad Abou-Chakra, Lingfeng Sun, Krishan Rana, Brandon May, Karl Schmeckpeper, Niko Suenderhauf, Maria Vittoria Minniti, Laura Herlant

Outline

This paper presents a novel approach to integrating simulation into a behavior replication pipeline, called 'real-is-sim'. Unlike existing real-data-only methods or simulation-to-real methods that struggle with simulation-to-real transfer, real-is-sim allows policies to seamlessly transition between real hardware and parallelized virtual environments. A dynamic digital twin based on an embedded Gaussian simulator is synchronized with the real world at 60Hz, and policies act only on the simulated robot and never on the real robot. During deployment, the real robot follows the joint states of the simulated robot, and the simulation is continuously updated with real-world measurements. This leaves the simulation-to-real transfer problem to the synchronization mechanism of the digital twin rather than the policy. We demonstrate real-is-sim on a long-duration manipulation task (PushT) and show that virtual evaluations are consistent with real results. We also compare how real data is augmented with virtual deployments and policies trained on different representations derived from the simulator state (including object positions, static and rendered images from robot-mounted cameras). The results highlight the flexibility of the real-is-sim framework.

Takeaways, Limitations

•
Takeaways:
◦
By enabling a seamless transition between simulation and the real world, it enables safe and efficient robotics policy development and deployment.
◦
Leverage dynamic digital twins to address simulation-to-real transition problems and improve the robustness of policy learning.
◦
Supports flexible and efficient policy learning by leveraging various simulator state representations.
◦
This increases the reliability of simulation-based robot policy development by demonstrating that evaluation results in a virtual environment are consistent with those in a real environment.
•
Limitations:
◦
It relies on the Embodied Gaussian simulator and requires verification of its scalability to other simulators.
◦
A sync rate of 60Hz may not be suitable for all tasks, and performance degradation may occur when higher frequency sync is required.
◦
The performance of the system heavily relies on accurate synchronization of digital twins, and any synchronization errors can negatively impact policy performance.
◦
The presented experiments are limited to a specific task (PushT), and further studies are needed to determine their generalizability to other types of tasks.
👍
Made with Slashpage