Large-scale language models (LLMs) are aligned to comply with safety guidelines by rejecting harmful instructions. A recent attack called 'abliteration' isolates and suppresses a single latent direction most responsible for the rejection behavior, allowing the model to generate unethical content. In this paper, we propose a defense technique that modifies the way the model generates rejections. We construct an extended rejection dataset that contains harmful prompts and full responses explaining the reasons for rejection. We then fine-tune Llama-2-7B-Chat and Qwen2.5-Instruct (with parameters 1.5B and 3B) on the extended rejection dataset, and evaluate the resulting system on a series of harmful prompts. Experimental results show that the extended rejection model reduces the rejection rate by up to 10%, while maintaining a high rejection rate, unlike the baseline model, which reduces the rejection rate by 70-80% after abliteration. Extensive evaluations on safety and usability demonstrate that extended rejection fine-tuning neutralizes abliteration attacks while maintaining general performance.
