This paper applies world models, widely used in robotics, games, and autonomous driving, to natural language processing, specifically, conversational systems. We build a conversational world model to predict user emotions, sentiments, intentions, and future utterances. We define a Partially Observable Markov Decision Process (POMDP) to model emotions, sentiments, and intentions as user beliefs, and propose a method to resolve information bottlenecks by maximizing them. Based on this user belief modeling, we apply a model-based reinforcement learning framework to the conversational system, presenting a novel framework called DreamCUB. Experimental results demonstrate that the pre-trained conversational world model achieves state-of-the-art performance in emotion classification and sentiment identification. Furthermore, combined training of the policy, critic, and conversational world models improves conversational quality. Further analysis demonstrates that the proposed method maintains an appropriate exploration-exploitation balance and demonstrates excellent transferability to non-domain scenarios, such as empathic conversations.
