This page organizes papers related to artificial intelligence published around the world. This page is summarized using Google Gemini and is operated on a non-profit basis. The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.
We demonstrate that training a single d-dimensional steering vector per layer using reinforcement learning and fixing all default weights matches the accuracy of a fully RL-tuned inference model on a mathematical inference task. In a model with 8 billion parameters, this adds only about 0.0016% of additional parameters, and reproduces the performance across various baseline models and mathematical inference benchmarks.
Takeaways, Limitations
•
We have narrowed the upper bound on the parameter budget required for higher-level thought process inference.
•
This suggests that millions of adapter weights are unnecessary.
•
The minimum learnable space reduces optimizer memory and inter-GPU communication, lowering the overall cost of fine-tuning.
•
Logit-lens analysis shows that the learned vectors amplify consistent token orientations, providing clearer insight into the model's internal computations.
•
The specific Limitations of the paper was not presented.
