This page organizes papers related to artificial intelligence published around the world. This page is summarized using Google Gemini and is operated on a non-profit basis. The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.
Distribution-Aligned Decoding for Efficient LLM Task Adaptation
Created by
Haebom
Author
Senkang Hu, Xudong Han, Jinqi Jiang, Yihang Tao, Zihan Fang, Yong Dai, Sam Tak Wu Kwong, Yuguang Fang
Outline
To reduce the cost of applying large-scale language models to downstream tasks, this paper proposes Steering Vector Decoding (SVDecode), a lightweight technique compatible with Parameter-Efficient Fine-Tuning (PEFT). SVDecode reinterprets task adaptation as output distribution alignment, directly controlling the output distribution during decoding rather than indirectly accessing the task distribution through weight updates. It extracts task-aware steering vectors from the Kullback-Leibler (KL) divergence gradients obtained through short warm-up fine-tuning and uses these to guide the model's output distribution toward the task distribution. SVDecode is linearly equivalent to the gradient step of full fine-tuning and yields a globally optimal solution for steering vector strengths. Across three tasks and nine benchmarks, SVDecode improves multiple-choice accuracy by up to 5% and open truth by 2% when used with four standard PEFT methods. On the common-sense dataset, it achieves similar gains (1-2%) without additional learnable parameters other than the PEFT adapter.
Takeaways, Limitations
•
Takeaways:
◦
SVDecode is a lightweight technique compatible with PEFT that improves the task adaptation performance of large-scale language models.
◦
It has a theoretical basis and performs similarly to full fine-tuning.
◦
Demonstrated performance improvements across a variety of tasks and benchmarks.
◦
Achieve performance improvements without additional learnable parameters.
