Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms

Created by
  • Haebom

Author

Bingxin Xu, Zhen Dong, Oussama Elachqar, Yuzhang Shang

Outline

This paper proposes ButterflyQuant, a novel quantization technique that addresses the performance degradation caused by activation outliers in 2-bit quantization. While existing rotation-based methods (QuIP, QuaRot) utilize a fixed Hadamard transform, this paper finds that each layer of a Transformer exhibits distinct outlier patterns. Therefore, we propose ButterflyQuant, a learnable Butterfly transform that adaptively rotates layers. The Butterfly transform is differentiable using continuous Givens rotation angles as parameters, guarantees orthogonality, and has a computational complexity of $O(n \log n)$ with only $\frac{n \log n}{2}$ learnable parameters. Furthermore, we introduce uniform regularization of activations after the transformation to ensure a smooth distribution suitable for quantization. Experimental results using 2-bit quantization on the LLaMA-2-7B model show that ButterflyQuant significantly outperforms QuaRot.

Takeaways, Limitations

•
Takeaways:
◦
Adaptive rotation across layers enables more efficient 2-bit quantization than conventional fixed transform-based methods.
◦
We present an efficient quantization method that guarantees differentiability and orthogonality using the learnable Butterfly transform.
◦
A small number of calibration samples and fast convergence time increase practical applicability.
◦
LLaMA-2-7B experimental results demonstrate superior performance over existing methods.
•
Limitations:
◦
ButterflyQuant's performance improvements may be limited to a specific model (LLaMA-2-7B) and 2-bit quantization.
◦
Generalization performance verification is needed for other types of large-scale language models or quantization bit counts.
◦
Further analysis of the effects of uniformity normalization may be required.
◦
Further research is needed on generalization performance for extreme low-bit quantization.
👍
Made with Slashpage