Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Improved Supervised Fine-Tuning for Large Language Models to Mitigate Catastrophic Forgetting

Created by
  • Haebom

Author

Fei Ding, Baiqiao Wang

Outline

In this paper, we present a novel method to address the catastrophic forgetting problem that occurs in supervised fine-tuning (SFT) methods for improving the instruction-following ability of open-source large-scale language models (LLMs). Without access to the original SFT data, we reconstruct the instruction distribution of the base model and synthesize a high-quality general-purpose dataset through a multi-model generation and filtering pipeline. By mixing this synthetic dataset with new domain-specific data and fine-tuning it, we experimentally demonstrate that it improves the performance of specific tasks without degrading the performance in the general domain.

Takeaways, Limitations

•
Takeaways:
◦
We present a novel SFT method that effectively alleviates the catastrophic forgetting problem without accessing existing SFT data.
◦
We demonstrate that leveraging synthetic datasets can improve the performance of LLM cost-effectively.
◦
Achieve both general domain performance and specific task performance improvements.
◦
Presenting the possibility of increasing the practicality of open source LLM.
•
Limitations:
◦
The quality of synthetic datasets may be lower than that of real datasets.
◦
Further validation of the generalization performance of the proposed method is needed.
◦
Possible dependencies on specific domains and models.
◦
The complexity and computational cost of the synthetic data generation process.
👍
Made with Slashpage