Although large-scale language models (LLMs) have made remarkable progress in mathematical inference, they still struggle with high-precision tasks such as numerical computation and formal symbolic manipulation. Integrating external tools has emerged as a promising approach to address this gap. Existing methods face three major challenges: constructing tool-integrated inference data, performing fine-tuning optimization, and improving inference. To overcome these limitations, we propose Tool-Integrated Hierarchical Optimization via RL (THOR). First, we build a high-quality tool-integrated inference path dataset using TIRGen, aligning and generalizing policies across diverse models. Second, we introduce an RL strategy that jointly optimizes episode-level problem solving and step-by-step code generation to perform fine-tuning hierarchical optimization. This is based on the core insight that the success of intermediate tool invocations is a strong predictor of the accuracy of the final solution. Finally, THOR incorporates a self-correction mechanism that utilizes immediate tool feedback to dynamically correct erroneous inference paths during the inference process. THOR demonstrates strong generalization across a wide range of models and operates effectively on both inference and non-inference models. It also achieves state-of-the-art performance on models of similar scale across multiple mathematical benchmarks and consistently delivers improvements across code benchmarks.
