This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
This paper presents a solution to the problem of equipping a quadruped robot with scalable autonomous and multi-purpose manipulation skills that demonstrate impressive locomotion capabilities in diverse environments. We introduce a cross-implementation imitation learning system that leverages data collected from a human and a quadruped robot with multiple manipulation modes, LocoMan. We develop a teleoperation and data acquisition pipeline that integrates and modularizes the observation and action spaces of humans and robots, and propose an efficient modular architecture that supports joint learning and pre-training using structured mode-aligned data across multiple implementations. We also construct the first manipulation dataset for LocoMan robots that covers a variety of household tasks in both single- and dual-handed modes, complementing the corresponding human dataset. Our validation results on six real-world manipulation tasks show an overall improvement of 41.9% over the baseline, and an average success rate of 79.7% in the out-of-distribution setting. Pre-training with human data contributes an overall improvement of 38.6% and an 82.7% success rate improvement in the out-of-distribution setting, consistently achieving better performance with only half the robot data. The code, hardware, and data have been made public ( https://human2bots.github.io ).
We significantly improved the manipulation performance of quadruped robots through a cross-implementation imitation learning system that integrates and leverages human and robot data.
◦
We improved data efficiency through efficient modular architecture and pre-training strategy.
◦
We show that pre-training on human data can improve performance while reducing the amount of robot data.
◦
We present successful validation results for real-world tasks.
◦
We made our research reproducible and scalable by making our code, hardware, and data open.
•
Limitations:
◦
Since we currently use a dataset limited to home tasks, generalization performance evaluation for various environments and tasks is required.
◦
Performance may be affected by the quality of the human demo.
◦
Further research is needed into robots' ability to cope with unpredictable situations and obstacles.
◦
Further research is needed to learn more complex and diverse manipulation skills.
