Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Are LLMs Enough for Hyperpartisan, Fake, Polarized and Harmful Content Detection? Evaluating In-Context Learning vs. Fine-tuning

Created by
  • Haebom

Author

Michele Joshua Maggini, Dhia Merzougui, Rabiraj Bandyopadhyay, Ga el Dias, Fabrice Maurel, Pablo Gamallo

Outline

This paper comprehensively evaluates the performance of large-scale language models (LLMs), which are attracting attention as a countermeasure against the proliferation of fake news, extreme bias, and harmful content on online platforms, across various models, usage methods, and languages. Using ten datasets and five languages (English, Spanish, Portuguese, Arabic, and Bulgarian), we experimentally compared and analyzed LLM adaptation paradigms in binary and multi-class scenarios. We tested various in-context learning strategies, including parameter-efficient fine-tuning, zero-shot prompts, codebooks, few-shot learning (including random selection and DPP-based multiple-choice examples), and chain-of-thought.

Takeaways, Limitations

Takeaways: This paper provides a comprehensive comparative analysis of various models, languages, and approaches for detecting fake news and harmful content using LLM. In particular, it highlights the importance of task-specific fine-tuning, even for small models, by demonstrating that fine-tuning outperforms in-context learning. Experimental results are presented, including the largest models, including LLaMA3.1-8b-Instruct, Mistral-Nemo-Instruct-2407, and Qwen2.5-7B-Instruct.
Limitations: The datasets and languages covered in this study may not encompass all possibilities. There is a possibility that the results may be biased toward certain languages or datasets. Further research is needed to determine generalizability to new types of fake news or harmful content.
👍