The inference performance of large-scale language models (LLMs) is significantly enhanced by reinforcement learning (RL), but the underlying parameter dynamics during RL training remain poorly understood. This study identifies two fundamental properties of RL-induced parameter updates in LLMs: (1) Rank-1 dominance, where the upper singular subspace of the parameter update matrix almost completely determines the inference improvement, recovering over 99% of the performance gain; and (2) Rank-1 linear dynamics, where the dominant subspace evolves linearly throughout training, enabling accurate predictions at initial checkpoints. Extensive experiments on eight LLMs and seven algorithms validate the generalizability of these properties. Building on these results, we propose AlphaRL, a plug-in acceleration framework that uses a short initial training period to extrapolate final parameter updates. This approach achieves up to 2.5x speedup while maintaining over 96% of the inference performance without requiring additional modules or hyperparameter tuning. It is a versatile and practical tool for large-scale RL and opens the way to a principled, interpretable, and efficient training paradigm for LLM.
