This paper proposes HAWKEYE, a novel post-training and inference framework, to address the efficiency challenges of Chain-of-Thought (CoT) inference. To address the computational cost and delay caused by excessive intermediate inference token generation in existing CoT inference, HAWKEYE adopts an approach where a large model generates concise CoT instructions, which are then utilized by a smaller model to generate responses. Reinforcement learning quantifies the redundancy of CoT inference and extracts dense information, significantly reducing token usage and computational costs while maintaining response quality. Experimental results demonstrate that HAWKEYE achieves similar response quality using only 35% of the total CoT, while improving clarity, consistency, and conciseness by approximately 10%. Furthermore, it speeds up inference by up to 3.4x and reduces inference costs by up to 60% on complex mathematical problems. HAWKEYE will be open-sourced.
