Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Beyond Code: The Multidimensional Impacts of Large Language Models in Software Development

Created by
  • Haebom

Author

Sardar Fatooreh Bonabi, Sarah Bana, Tingting Nian, Vijay Gurbaxani

Outline

This paper empirically analyzes the impact of large-scale language models (LLMs) on open source software (OSS) development. We analyze data from 88,022 GitHub OSS developers in Italy, France, and Portugal, using the Italian ChatGPT temporary ban as a natural experiment to examine how LLMs affect OSS developers’ work in three key areas: code development, collaborative knowledge transfer, and technology development. We use the difference-in-differences (DDI) estimation method to measure the impact of LLM accessibility on developer productivity, knowledge sharing, and technology acquisition.

Takeaways, Limitations

Takeaways:
LLM accessibility improves developer productivity by 6.4%, knowledge sharing by 9.6%, and skill acquisition by 8.4%.
The benefits of LLM vary depending on the level of user experience. Novice developers benefit more from increased productivity, while experienced developers benefit more from improved knowledge sharing and skill acquisition.
LLM-supported learning is most effective in contexts that are technically complex, fragmented or rapidly changing.
The productivity benefits of an LLM extend beyond direct code creation to improved collaborative learning and knowledge exchange among developers.
Strategically deploying an LLM can accelerate the onboarding and productivity of entry-level developers, enhance knowledge sharing and collaboration among mid-level developers, and support rapid skill acquisition, improving long-term organizational productivity and agility.
Limitations:
Since we used the temporary blocking of ChatGPT in Italy as a natural experiment, it is difficult to completely rule out the influence of other factors.
The generalizability of the results is limited because the countries analyzed are limited to Italy, France, and Portugal.
There is a lack of analysis of the differences in impact by type and use of LLM.
👍