This paper proposes GOAT (GFlOwNet-guided Distribution Alignment), a novel method for addressing hallucinations in language model-based text-to-speech (TTS) systems. Unlike existing methods, GOAT is a post-training framework that mitigates hallucinations without excessive training resources or inference delays. We analyze the strong correlation between model uncertainty and hallucinations and reframe TTS generation as a trajectory flow optimization problem, employing enhanced sub-trajectory balance objectives and sharply tuned internal rewards as the target distribution. We integrate reward temperature reduction and learning rate optimization to balance stability and performance. Experimental results demonstrate excellent generalization and effectiveness, reducing character error rates by more than 50% and uncertainty by up to 58% on challenging test cases.
