Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

A comprehensive study of LLM-based argument classification: from LLAMA through GPT-4o to Deepseek-R1

Created by
  • Haebom

Author

Marcin Pietro n, Rafa{\l} Olszowski, Jakub Gomu{\l}ka, Filip Gampel, Andrzej Tomski

Outline

This paper evaluates the performance of large-scale language models (LLMs) in the field of argument mining (AM) using various datasets (Args.me, UKP, etc.). By comparing and analyzing several LLMs such as GPT, Llama, and DeepSeek and inference-enhanced variants using Chain-of-Thoughts algorithms, ChatGPT-4o showed the best performance in general argument classification benchmarks, and Deepseek-R1 showed the best performance among models with added inference functions. However, even the best-performing models made errors, and we analyzed the types of such errors and suggested future directions for improvement. In addition, we pointed out __T80991__ of the existing prompt algorithm and presented an in-depth analysis of the shortcomings of the argument dataset used. This study is evaluated as the first extensive analysis of Args.me and UKP datasets using LLM and prompt algorithms.

Takeaways, Limitations

•
Takeaways:
◦
We confirmed the superiority of ChatGPT-4o and Deepseek-R1 through comparative analysis of argument mining performance using various LLM and prompt algorithms.
◦
The current status and limitations of LLM-based argument mining were clearly presented, suggesting future research directions.
◦
Analysis of the Args.me and UKP datasets suggests directions for improvement of the datasets themselves.
◦
We revealed the Limitations of the existing prompt algorithm and suggested a direction for improvement.
•
Limitations:
◦
The types of LLMs and datasets used in the analysis may be limited.
◦
The analysis of LLM's error types needs to be more in-depth.
◦
There is a lack of experimental verification of the proposed improvement directions.
👍
Made with Slashpage