This paper proposes a method to distill the utility judgment capabilities of a large-scale language model (LLM) into a smaller model to enhance the efficiency of utility-based retrieval in Augmented Search Generation (RAG). Specifically, we apply a utility-based selection approach to generate answers to complex questions, enabling dynamic paragraph selection rather than a fixed threshold. By training student models (RankQwen1.7B, UtilityQwen1.7B) that learn fake answer generation and utility judgment from a teacher LLM (Qwen3-32B), we improve answer quality while reducing computational costs. We plan to release relevance ranking and utility-based selection annotations for the MS MARCO dataset to support research in this area.
