In this paper, we present the EFRame framework, which improves the Group Relative Policy Optimization (GRPO), an efficient reinforcement learning algorithm, to solve the problems of limited exploration, low sample efficiency, and instability that hinder performance in complex inference tasks. EFRame systematically integrates three core elements of exploration, filtering, and experience replay to perform high-quality trajectory exploration, low-quality sample removal, and iterative utilization of rare but informative samples. This builds a stable learning cycle and structures the transition process from exploration to convergence, thereby improving the model’s inference ability. Through various inference benchmark experiments, we demonstrate that EFRame not only improves the robustness and efficiency of learning, but also enables deep inference capabilities that could not be achieved with conventional GRPO. In addition, it provides deep insights into the contribution of each sample through fine-grained classification of training samples, and provides an efficient and precise entropy control mechanism that is important for balancing exploration and convergence.
