This paper introduces Neon (Negative Extrapolation from Self-Training), a novel training method proposed to address the lack of high-quality training data. Neon addresses the problem of model self-feeding disorder (MAD), which arises during model fine-tuning using self-synthesized data from a generative model. Neon augments the weights of the trained model through reverse gradient updating to improve model performance. Neon is applicable to a variety of architectures and datasets and demonstrates excellent performance while consuming minimal additional training resources.
