[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

The GPT Surprise: Offering Large Language Model Chat in a Massive Coding Class Reduced Engagement but Increased Adopters Exam Performances

Created by
  • Haebom

Author

Allen Nie, Yash Chandak, Miroslav Suzara, Ali Malik, Juliette Woodrow, Matt Peng, Mehran Sahami, Emma Brunskill, Chris Piech

Outline

This paper presents the results of a large-scale randomized controlled trial that investigated the impact of accessibility of a GPT-4-based chat interface on coding education with 5,831 students worldwide. Providing GPT-4 access to some students improved test scores for students who used the tool, but overall, advertising GPT-4 decreased test participation. Other forms of class participation were also reduced, but these decreases varied by student country. Providing LLM access to students from low-HDI countries increased test participation. In conclusion, we conclude that the positive effects of LLM use in introductory coding classes coexist with the negative effects of decreased engagement, and the effects on long-term academic achievement are uncertain. This highlights the need for further research on the integration of LLMs into educational settings.

Takeaways, Limitations

Takeaways:
LLMs like GPT-4 show promise in improving students' test scores in coding education.
Effective in increasing exam participation among students from low-HDI countries with low LLM accessibility.
Suggesting that the introduction of LLMs may have a negative impact on student engagement.
Highlights the need for further research into the integration of LLM into the educational environment.
Limitations:
Because this study was conducted in a specific coding class environment, it is difficult to generalize to other educational environments.
Further research is needed on how GPT-4 can be used and how to leverage it effectively.
Further research is needed on the long-term impact of LLM introduction on student engagement.
There are limitations in analyzing differences between countries based on the Human Development Index.
👍