Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Industrial LLM-based Code Optimization under Regulation: A Mixture-of-Agents Approach

Created by
  • Haebom

Author

Mari Ashiga, Vardan Voskanyan, Fateme Dinmohammadi, Jingzhi Gong, Paul Brookes, Matthew Truscott, Rafail Giavrimis, Mike Basios, Leslie Kanthan, Wei Jie

Outline

This paper presents a mixed-agent (MoA) approach for code optimization in regulated industries. In environments where the use of commercial LLMs is limited due to regulatory compliance and data privacy concerns, we propose MoA, which combines multiple specialized open-source LLMs to generate code. We compare it with TurinTech AI's genetic algorithm (GA)-based ensemble system and individual LLM optimizers. Experimental results using real-world industrial codebases demonstrate that MoA achieves cost savings of 14.3% and 22.2%, respectively, and optimization time reductions of 28.6% and 32.2%, respectively, using open-source models. Furthermore, we demonstrate the superiority of the GA-based ensemble on commercial models, and both ensembles outperform individual LLMs. We validate its applicability in real-world environments using 50 code fragments and seven LLM combinations. Ultimately, we provide practical guidance on balancing regulatory compliance and optimization performance.

Takeaways, Limitations

Takeaways:
We present the possibility of effective code optimization using open-source LLM-based MoA in a regulated industry environment.
Empirically demonstrating that MoA offers significant benefits in cost reduction and optimization time reduction.
Comparative analysis of the pros and cons of commercial and open-source LLMs to provide applicable guidelines for industrial settings.
Highly reliable results based on large-scale experiments using real industry codebases.
Limitations:
The types and versions of open source and commercial LLMs used in this study were not explicitly mentioned, limiting generalizability.
Since the performance of MoA may vary depending on the type and configuration of LLM used, further research on various combinations is needed.
Because the characteristics of the codebase used in the experiment are not clearly presented, generalizability to other types of codebases needs to be examined.
In the comparative analysis with GA-based ensemble systems, there is a lack of information on the specific settings of the GA system.
👍