This paper emphasizes the importance of training machine learning models to clearly understand the factors that define each class. Previous studies have focused on identifying spurious correlations in datasets by relying solely on data or error analysis, but have failed to detect spurious correlations learned by models that are not revealed by counterexamples in the validation or training sets. To overcome these limitations, this paper proposes WASP (Weight-space Approach to Detecting Spuriousness), a novel method that analyzes the model's weights, the decision-making mechanism, rather than analyzing the model's predictions. WASP analyzes how the base model's weights shift in a direction that captures various (spurious) correlations during fine-tuning on a specific dataset. Unlike previous studies, WASP (i) exposes spurious correlations in datasets that are not revealed by training or validation counterexamples, (ii) works across various modalities, such as images and text, and (iii) demonstrates its ability to detect previously unknown spurious correlations learned by the ImageNet-1k classifier.
