Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

DatawiseAgent: A Notebook-Centric LLM Agent Framework for Adaptive and Robust Data Science Automation

Created by
  • Haebom

Author

Ziming You, Yumiao Zhang, Dexuan Xu, Yiwei Lou, Yandong Yan, Wei Wang, Huaming Zhang, Yu Huang

Outline

DatawiseAgent is a notebook-centric LLM agent framework for data science automation. To overcome the narrow task scope, limited generalization, and dependency on state-of-the-art LLMs of existing LLM agents, we draw inspiration from the work of human data scientists and introduce a unified interaction representation and an FST-based multi-stage architecture. DatawiseAgent enables flexible long-term planning, incremental solution development, and recovery from execution failures, achieving state-of-the-art performance that surpasses robust foundational models like AutoGen and TaskWeaver across a variety of data science scenarios. Furthermore, it demonstrates robustness and scalability, demonstrating stable performance degradation even with weaker models.

Takeaways, Limitations

Takeaways:
Achieving SOTA performance in data science automation.
Maintains effective performance not only on strong base models but also on weak models.
Demonstrate the potential to solve complex data science problems through flexible long-term planning and incremental solution development.
Improved agent stability by enabling recovery from execution failures.
Limitations:
There is no specific mention of Limitations in the paper.
👍