This paper proposes a self-improving faith-aware contrastive tuning (SI FACT) framework to address the "knowledge conflict" problem, where large-scale language models (LLMs) rely on their internal parameter knowledge to ignore the context provided in knowledge-intensive tasks. SI FACT automatically generates high-quality, structured contrastive training data (anchor samples, semantically equivalent positive samples, and negative samples mimicking unfaithful scenarios) through a self-directed mechanism, significantly reducing the cost of manual annotation. The model is then trained by placing reliable responses close together and unfaithful responses farther apart through contrastive learning. Based on Llama3 8B Instruct, the SI FACT model improves contextual recall by 6.2% and significantly reduces internal memory dependency compared to the best baseline model on the ECARE KRE and COSE KRE benchmarks. This demonstrates that SI FACT is an effective, data-efficient, and practical method for building more proactive and reliable language models.
