This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
RALAD: Bridging the Real-to-Sim Domain Gap in Autonomous Driving with Retrieval-Augmented Learning
Created by
Haebom
Author
Jiacheng Zuo, Haibo Hu, Zikang Zhou, Yufei Cui, Ziquan Liu, Jianping Wang, Nan Guan, Jin Wang, Chun Jason Xue
Outline
To improve the robustness of autonomous driving systems, models trained on real datasets have difficulty adapting to new environments when faced with exceptional situations such as extreme weather conditions. Since it is difficult to collect such exceptional situations in real environments, simulators must be used for validation. However, the high computational cost and the domain gap in data distribution have made it difficult to smoothly transition between real and simulated driving scenarios. To address these issues, in this paper, we propose a novel framework, Retrieval-Augmented Learning for Autonomous Driving (RALAD), designed to bridge the gap between real and simulated environments at low cost. RALAD features three key design features: domain adaptation via an improved optimal transfer (OT) method that considers both individual and grouped image distances, a simple and unified framework that can be applied to various models, and an efficient fine-tuning technique that maintains robustness while fixing computationally expensive layers. Experimental results demonstrate that RALAD compensates for the performance degradation in simulated environments while maintaining the accuracy of real scenarios across three different models. Taking Cross View as an example, in real scenarios, the mIOU and mAP metrics remain stable before and after RALAD fine-tuning, while in simulation environments, the mIOU and mAP metrics are improved by 10.30% and 12.29%, respectively. In addition, the retraining cost of our approach is reduced by about 88.1%. The code can be found at https://github.com/JiachengZuo/RALAD.git .
Bridging the gap between reality and simulation through efficient domain adaptation using improved optimal transfer (OT) methods.
◦
It provides a simple, unified framework applicable to a variety of models.
◦
Enables efficient fine-tuning by pinning computationally expensive layers.
◦
Achieve improved performance (improved mIOU and mAP metrics) and reduced retraining costs in a simulation environment.
◦
Improves simulation environment performance without compromising real-world performance.
•
Limitations:
◦
Further research is needed to investigate the generalization performance of the proposed method. Extensive experiments on various environments and models are needed.
◦
Since the results are for a specific simulator and dataset, verification is needed to determine whether they can be generalized to other simulators or datasets.
◦
There may be a lack of detailed explanation on tuning the parameters of the Optimal Transfer (OT) method.
