In this paper, we propose a novel text detoxification method that removes harmful content while preserving the original meaning to address the problem of harmful content proliferation on social media. To overcome the limitations of existing methods, such as low performance, difficulty in preserving meaning, vulnerability to out-of-distribution data, and high data dependency, we present a two-stage learning framework. In the first stage, a strong initial model is built through supervised learning fine-tuning using high-quality filtered parallel data, and in the second stage, LLM is trained through Group Relative Policy Optimization using unlabeled harmful input data and a user-defined reward model. Experimental results show that the proposed method effectively alleviates the limitations of existing methods, achieves state-of-the-art performance, improves generalization performance, and significantly reduces the dependency on annotation data. The source code is available on GitHub.
