Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

CyberLLMInstruct: A Pseudo-malicious Dataset Revealing Safety-performance Trade-offs in Cyber Security LLM Fine-tuning

Created by
  • Haebom

Author

Adel ElZemity, Budi Arief, Shujun Li

Outline

This paper addresses the opportunities and security risks of integrating large-scale language models (LLMs) into cybersecurity applications. We present the CyberLLMInstruct dataset, consisting of 54,928 pseudo-malicious instruction-response pairs across cybersecurity tasks, including malware analysis, phishing simulations, and zero-day vulnerabilities. Comprehensive evaluations using seven open-source LLMs reveal that fine-tuning improves cybersecurity task performance (up to 92.50% accuracy achieved on CyberMetric), but reveals significant tradeoffs, including a significant decrease in security across all tested models and attack vectors (e.g., the security score of Llama 3.1 8B against prompt injection decreased from 0.95 to 0.15). The dataset comprehensively covers the cybersecurity landscape by incorporating diverse sources, including CTF challenges, academic papers, industry reports, and the CVE database. Our findings highlight the unique challenges of securing LLMs in hostile environments and demonstrate the importance of developing fine-tuned methodologies that balance performance enhancements with security-sensitive domains.

Takeaways, Limitations

•
Takeaways:
◦
Demonstrating the potential of LLM fine-tuning to improve performance for cybersecurity tasks.
◦
Clearly demonstrates the negative impact of LLM fine-tuning on safety.
◦
The need to develop a new fine-tuning methodology to ensure the safety of LLM in the field of cybersecurity is raised.
◦
Contribute to the advancement of LLM research in cybersecurity through the CyberLLMInstruct dataset.
•
Limitations:
◦
The type and version of LLM used may be limited.
◦
The types of attack vectors used in the evaluation may be limited.
◦
Further research is needed to explore the versatility and generalizability of the CyberLLMInstruct dataset.
◦
Lack of specific methods to find the optimal balance between safety and performance.
👍
Made with Slashpage