This is a page that curates AI-related papers published worldwide. All content here is summarized using Google Gemini and operated on a non-profit basis. Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.
Scaling Laws of Motion Forecasting and Planning - Technical Report
Created by
Haebom
Author
Mustafa Baniodeh, Kratarth Goel, Scott Ettinger, Carlos Fuertes, Ari Seff, Tim Shen, Cole Gulino, Chenjie Yang, Ghassen Jerfel, Dokook Choe, Rui Wang, Benjamin Charrow, Vinutha Kallem, Sergio Casas, Rami Al-Rfou, Benjamin Sapp, Dragomir Anguelov
Outline
We studied the empirical scaling laws of the encoder-decoder autoregressive Transformer model family for joint motion prediction and planning tasks in autonomous driving. Using a 500,000-hour driving dataset, we show that model performance improves as a power-law function of the total compute budget, similar to language modeling, and that there is a strong correlation between model training loss and model evaluation metrics. Most interestingly, closed-loop metrics also improve with scaling, which has important implications for the suitability of open-loop metrics for model development and bottom-up approaches. We also studied the optimal scaling of the number of Transformer parameters and training data size for models optimized for training compute. We found that optimal scaling requires increasing model size 1.5 times faster than the dataset size as the training compute budget increases. We also studied inference time computing scaling, showing that sampling and clustering the outputs of smaller models makes them competitive with larger models, and beyond a crossover point, larger models achieve higher inference compute efficiency. Overall, the experimental results demonstrate that optimizing the training and inference time scaling characteristics of motion prediction and planning models is a key means of improving performance across a variety of driving scenarios. Finally, we briefly explore the utility of using recorded driving data from other agents to improve the performance of self-agents, a crucial area of research addressing the lack of robotics data for large-scale model training.
Takeaways, Limitations
•
Takeaways:
◦
We empirically show that model performance improves as a power-law function of the total computing budget.
◦
A strong correlation was found between model training loss and model evaluation metrics.
◦
We also confirm that the closed-loop indicator improves with scaling, suggesting the limitations of the open-loop indicator.
◦
We present optimal scaling strategies for model size and dataset size to optimize training computation.
◦
Suggesting the possibility of improving inference time computing efficiency through output sampling and clustering of small models.
◦
Suggesting the possibility of improving the performance of self-agents by utilizing data from other agents.
•
Limitations:
◦
The research subject is limited to a specific series of transformer models.
◦
Using a 500,000-hour driving data set, further research is needed to determine the generalizability of the data.
◦
Further analysis and explanation of the closed-loop indicator improvement is needed.
◦
Further validation of generalization performance across various driving scenarios is needed.
◦
More in-depth research is needed on how other agents leverage data.
