This paper introduces EAPrivacy, an evaluation benchmark for measuring the privacy awareness of embodied agents leveraging large-scale language models (LLMs) in the physical world. EAPrivacy uses four procedurally generated scenarios to test their ability to handle sensitive objects, adapt to changing environments, respect privacy constraints, and resolve conflicts with social norms. Our results show that even the top-performing model, Gemini 2.5 Pro, achieved 59% accuracy in changing physical environment scenarios, and prioritized task completion over constraints in up to 86% of privacy-relevant situations. Leading models, such as GPT-4o and Claude-3.5-haiku, ignored social norms more than 15% of the time in situations where privacy conflicted with important social norms. These results reveal a fundamental inconsistency in the current LLM's ability to physically embody privacy and highlight the need for more robust, physically aware alignment.
