Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

In-context learning for the classification of manipulation techniques in phishing emails

Created by
  • Haebom

Author

Antony Dalmiere (LAAS-TRUST, LAAS), Guillaume Auriol (LAAS-TRUST, INSA Toulouse), Vincent Nicomette (LAAS-TSF, LAAS), Pascal Marchand (LERASS)

Outline

This paper proposes a phishing email analysis method using the context-in-context learning (ICL) of a large-scale language model (LLM), considering the psychological manipulation factor, which is the limitation of existing phishing detection methods. Based on a classification system of 40 psychological manipulation techniques, we performed small-shot learning using the GPT-4o-mini model and a real French phishing email dataset (SignalSpam). As a result of evaluation using a test set manually annotated by 100 people, we confirmed that the main techniques such as Baiting, Curiosity Appeal, and Request for Minor Favor were effectively identified with an accuracy of 0.76. This provides the possibility of sophisticated phishing analysis using ICL and insight into attacker strategies.

Takeaways, Limitations

•
Takeaways:
◦
We demonstrate that the ICL of LLM can be used to analyze and refine the psychological manipulation techniques in phishing emails.
◦
Offers new possibilities for more sophisticated phishing detection and attacker strategy analysis than traditional methods.
◦
We confirmed that LLMs such as GPT-4o-mini show high performance with small data.
•
Limitations:
◦
The size of the test dataset is limited (100 emails).
◦
Validation of generalizability is needed using only the French phishing email dataset.
◦
An accuracy of 0.76 is high, but higher accuracy may be required for real-world service applications.
◦
Further research is needed on different languages and phishing techniques.
👍
Made with Slashpage