Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

LLM Test Generation via Iterative Hybrid Program Analysis

Created by
  • Haebom

Author

Sijia Gu, Noor Nashid, Ali Mesbah

Outline

This paper presents the Panta technique to address the challenges of automating unit test generation for complex methods in real-world projects. Panta overcomes the limited control flow structure inference capability of LLM by mimicking the iterative process of developers analyzing code and constructing test cases. By integrating static control flow analysis and dynamic code coverage analysis, LLM systematically guides the generation of better test cases by identifying undiscovered execution paths. An iterative feedback-based mechanism continuously improves test generation based on insights from static and dynamic path coverage, ensuring more comprehensive and effective testing. Experimental evaluations on classes with high cyclomatic complexity from open source projects show that Panta achieves 26% better line coverage and 23% better branch coverage than state-of-the-art techniques.

Takeaways, Limitations

Takeaways:
We present Panta, a new technique for improving the efficiency of automatic unit test generation using LLM.
Addressing the low branch coverage problem of traditional LLM-based test generation (Limitations) by integrating static and dynamic analysis.
Validation of Panta's superiority through experimental results targeting actual open source projects.
Improve the accuracy and coverage of test generation through an iterative feedback mechanism.
Limitations:
Panta's performance is limited to evaluation results for a specific class of open source projects. Further research is needed to determine its generalizability across a wider range of projects and code styles.
Focusing on classes with high cyclomatic complexity, its effectiveness on classes with low complexity is unclear.
It may depend on the performance of LLM, and its performance may change as LLM develops.
Further research is needed on applicability and scalability in real industrial environments.
👍