This paper focuses on visual personalization, a crucial area in user-centric AI systems. We introduce MMPB, the first comprehensive benchmark for evaluating the personalization capabilities of large-scale Vision-Language Models (VLMs). MMPB consists of 10,000 image-query pairs and 111 personalizable concepts across four categories (human, animal, object, and character), with preference-based queries included in the human category. We structure personalization into three main task types to evaluate the performance of 23 widely used VLMs and find that most VLMs struggle with personalization.