This paper addresses the problem that psychologically persuasive phishing emails generated using large-scale language models (LLMs) can evade existing detection systems. To address this issue, we propose PiMRef, a reference-based phishing email detector that leverages knowledge-based invariants. PiMRef extracts the alleged identity of a sender from an email, verifies the legitimacy of a domain using a predefined knowledge base, and detects call-to-action prompts that encourage user engagement. Contradictory claims are flagged as phishing indicators and provided with human-readable explanations. It demonstrates higher precision (8.8% improvement) and efficiency than existing methods, and achieves excellent performance (precision 92.1%, recall 87.9%, and average execution time 0.05 s) in real-world evaluations.