This paper addresses the lack of a systematic understanding of effective control mechanisms for large-scale language models (LLMs) that employ hierarchical instruction hierarchies (e.g., system-level instructions take precedence over user messages). We present a systematic evaluation framework based on constraint prioritization to assess how well LLMs enforce instruction hierarchies. Experiments on six state-of-the-art LLMs reveal that the models struggle to consistently enforce instruction prioritization even in simple formal conflicts. The widely used system/user prompt separation fails to establish a reliable instruction hierarchy, and the models exhibit strong inherent biases toward certain constraint types, regardless of prioritization. LLMs tend to more reliably follow constraints constructed through natural social hierarchies (e.g., authority, expertise, consensus) than system/user roles, suggesting that pre-trained social structures can act as potential control priors, exerting a stronger influence than post-training safeguards.
