[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

A Study of LLMs' Preferences for Libraries and Programming Languages

Created by
  • Haebom

Author

Lukas Twist, Jie M. Zhang, Mark Harman, Don Syme, Joost Noppen, Helen Yannakoudakis, Detlef Nauck

Outline

This paper presents the first empirical study of library and programming language preferences for code generation across eight diverse large-scale language models (LLMs). We find that LLMs tend to overuse popular libraries like NumPy (up to 48% of the time, they use them unnecessarily) and favor Python as their primary language (58% of the time, even for high-performance project initialization tasks where Python is not the optimal language, and Rust is never used). This is because LLMs prioritize familiarity and popularity over suitability and task-specific optimality, which can lead to security vulnerabilities and technical debt, and limit exposure to newly developed, more suitable tools and languages. Understanding and addressing these biases is therefore essential for responsibly integrating LLMs into software development workflows.

Takeaways, Limitations

Takeaways:
We first empirically demonstrate that LLM has biases towards specific libraries and programming languages when generating code.
Suggesting that the choice of libraries and languages for an LLM can have a significant impact on software development practices.
LLM bias raises the possibility of security vulnerabilities and technical debt.
Emphasizes the importance of addressing bias and considering appropriateness for responsible use of LLMs.
Limitations:
The number of LLMs included in the study may be limited.
Further research is needed on generalizability to different types of code generation tasks.
Lack of specific methodologies to address LLM bias.
👍