PLUS (Preference Learning Using Summarization) is a novel framework developed for personalized responses from LLM AI assistants. It overcomes the limitations of reinforcement learning from human feedback (RLHF) and generates personalized responses for each user by summarizing each user's preferences, characteristics, and past conversations. PLUS operates through an online co-adaptation loop that simultaneously trains a user summary model and a reward model. It delivers robust performance on new users and conversation topics, zero-shot personalization comparable to models like GPT-4, flexible user context learning, and interpretable user representations.
