To address model hallucination in Retrieval-Augmented Generation (RAG) systems, we propose Self-Supervised Faithfulness Optimization (SSFO). SSFO constructs preference data pairs by contrasting model outputs with and without context, and uses Direct Preference Optimization (DPO) to improve model accuracy without labeling costs or additional inference overhead. SSFO leverages the "likelihood displacement" phenomenon, which transfers probability mass to context-aligned tokens, and proposes an improved DPO loss function based on this. SSFO achieves state-of-the-art performance on multiple datasets, outperforming existing methods and preserving generalization and direction-following capabilities in multilingual environments.
