This paper proposes Merge-of-Thought Distillation (MoT), a novel method for efficiently distilling the inference capabilities of long-range thought processes (CoT) models by leveraging multiple teacher models. To overcome the limitations of conventional distillation methods that rely on a single teacher model, MoT trains a student model by integrating the guidance from multiple teacher models. This iterative process involves fine-tuning the student model for each teacher model and merging the results in the weight space. Applying MoT to the Qwen3-14B student model on a competitive mathematics benchmark using only a small number of high-quality CoT samples, the method outperforms powerful models such as DEEPSEEK-R1, QWEN3-30B-A3B, QWEN3-32B, and OPENAI-O1. MoT outperforms single-teacher distillation and simple multi-teacher integration methods, mitigating overfitting and demonstrating robustness to distribution shift and equally high-quality teacher models. Furthermore, it reduces catastrophic forgetting, enhances general reasoning beyond the mathematical domain, and cultivates better teacher models. These results demonstrate that MoT is a simple and scalable method for efficiently distilling long-range CoT capabilities from diverse teacher models into small student models.
