This paper introduces Middo, a framework for dynamically optimizing training data for Supervised Fine-Tuning (SFT) large-scale language models (LLMs). Middo aims to continuously evolve data to improve model performance, leveraging model-aware data selection and context-preserving data refinement. It identifies inappropriate samples through triaxial model signals (loss patterns, embedding cluster dynamics, and self-alignment scores) and improves them while preserving semantic integrity using an adaptive optimization engine. This framework presents a new paradigm for sustainable LLM training through dynamic co-evolution of data and model. Experimental results demonstrate that Middo achieves an average accuracy improvement of 7.15%, demonstrating superior performance even when maintaining the original dataset size.
