Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Performance of LLMs on Stochastic Modeling Operations Research Problems: From Theory to Practice

Created by
  • Haebom

Author

Akshit Kumar, Tianyi Peng, Yuhang Wu, Assaf Zeevi

Outline

This paper is the first to evaluate the ability of large-scale language models (LLMs) to solve operational research (OR) problems, specifically probabilistic modeling problems characterized by uncertainty using probability, statistics, and stochastic process tools. We assessed the problem-solving ability of LLMs by manually collecting graduate-level assignments and PhD exam questions, and investigated their real-world decision-making ability under uncertainty using SimOpt, an open-source simulation optimization library. Our results show that while state-of-the-art LLMs demonstrate human-expert-level proficiency in both classroom and real-world settings, significant additional work is needed to reliably automate the probabilistic modeling pipeline. This study highlights the potential for building AI agents that can support OR researchers and amplify the real-world impact of OR through automation.

Takeaways, Limitations

•
Takeaways:
◦
State-of-the-art LLM achieves human expert-level performance in solving probabilistic modeling problems.
◦
Presenting the possibility of automating OR problem solving using LLM.
◦
Support OR researchers and identify opportunities to amplify the real-world impact of OR.
•
Limitations:
◦
Further research is needed to achieve robust automation of probabilistic modeling pipelines.
◦
Further validation of the representativeness of the problems used in the study is needed.
◦
Further research is needed on the transparency and explainability of the LLM resolution process.
👍
Made with Slashpage