Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

GenBFA: An Evolutionary Optimization Approach to Bit-Flip Attacks on LLMs

Created by
  • Haebom

Author

Sanjay Das, Swastik Bhattacharya, Souvik Kundu, Shamik Kundu, Anand Menon, Arnab Raha, Kanad Basu

Outline

This paper addresses the vulnerability of large-scale language models (LLMs) to hardware-based threats, especially bit-flip attacks (BFAs). While previous studies have argued that transformer-based architectures are more robust against BFAs, this paper shows that even a few bit-flips can severely degrade the performance of LLMs. To this end, we propose AttentionBreaker, a novel framework that efficiently explores the parameter space of LLMs to identify important parameters. In addition, we present GenBFA, an evolutionary optimization strategy that finds the most important bits and improves the attack efficiency. Experimental results show that even a few bit-flips can drastically degrade the performance of LLMs. For example, in the LLaMA3-8B-Instruct model, the accuracy of MMLU tasks drops from 67.3% to 0%, and the perplexity of Wikitext jumps from 12.6 to 4.72 x 10^5 with just three bit-flips. This highlights the effectiveness of AttentionBreaker and the vulnerability of LLM architectures.

Takeaways, Limitations

•
Takeaways:
◦
We show that LLM is highly vulnerable to BFA, and even a small number of bit flips can lead to catastrophic performance degradation.
◦
AttentionBreaker and GenBFA present a novel way to efficiently find important parameters of LLM and perform attacks.
◦
We present new research directions for enhancing the security of LLM.
◦
Raise awareness of LLM security threats in real-world environments.
•
Limitations:
◦
The currently proposed method only presents results for a specific LLM architecture and 8-bit quantized models. Further research is needed for other architectures or precisions.
◦
The effectiveness of AttentionBreaker and GenBFA may vary with model size, and their applicability to large-scale LLMs needs to be further verified.
◦
There is a lack of research on effective defensive techniques in real-world attack environments.
◦
A comprehensive analysis of various attack vectors and defensive techniques is required.
👍
Made with Slashpage