This paper explores the application of world models to natural language processing tasks, focusing specifically on conversational systems. We build a conversational world model that predicts user emotions, sentiments, intentions, and future utterances. We define a POMDP (Property-Oriented Model of Mind Processing) to demonstrate that emotions, sentiments, and intentions can be modeled as user beliefs, arguing that information bottlenecks can be resolved by maximizing the information bottleneck. Using this user belief modeling, we apply a model-based reinforcement learning framework to the conversational system and propose a framework called DreamCUB. Experiments demonstrate that a pre-trained conversational world model achieves state-of-the-art performance in sentiment classification and sentiment identification, and joint training of the policy, critic, and conversational world models further enhances conversational quality. Furthermore, we demonstrate that the model maintains a reasonable exploration-exploitation balance and transfers well to non-domain scenarios, such as empathic conversations.
