This paper proposes CostFilter-AD, which introduces the cost filtering concept borrowed from classical matching tasks such as depth and flow estimation, to the UAD problem to address the problem of inaccuracy in the process of deriving anomaly scores by relying on image- or feature-level matching in existing unsupervised anomaly detection (UAD) methods. CostFilter-AD constructs a matching cost volume between input and normal samples, and suppresses matching noise while preserving edge structure and capturing subtle anomalies through a cost volume filtering network guided by input observations. It is designed as a generic post-processing plugin that can be integrated into both reconstruction-based and embedding-based methods. Extensive experiments on the MVTec-AD and VisA benchmarks demonstrate the general advantages of CostFilter-AD for both single- and multi-class UAD tasks.