This paper focuses on steering, a promising approach for controlling the parameters of large-scale language models (LLMs) without modifying them. Existing steering methods rely on large datasets to learn clear action information, but steering vectors learned from small datasets often contain task-irrelevant noisy features, resulting in ineffectiveness. To address this, this paper proposes SAE-RSV (Refinement of Steering Vector via Sparse Autoencoder), which semantically denoises and augments steering vectors using a sparse autoencoder (SAE). SAE removes task-irrelevant features and augments missing task-relevant features from small datasets based on semantic similarity with identified relevant features. Experimental results demonstrate that the proposed SAE-RSV outperforms all baseline methods, including supervised learning-based fine-tuning. We demonstrate that refining the original steering vector via SAE allows for the construction of effective steering vectors from limited training data.
