In this paper, we propose Savage Conversation Forests (SCF), a reinforcement learning framework for fine-tuning large-scale language models (LLMs) on multi-round conversation tasks. Existing methods such as DPO and GRPO are effective for single-round tasks, but they are not suitable for multi-round tasks such as medical diagnostic interviews where the initial conversation rounds affect the results. SCF generates multiple possible conversation continuations for each round, allowing the model to learn how the initial responses affect the subsequent interactions and diagnosis results. In simulation experiments on doctor-patient conversations, SCF achieves higher diagnostic accuracy than linear conversation structures, suggesting that the branched training structure is an important strategy for fine-tuning LLMs on complex multi-round conversation tasks.
