Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Do Concept Bottleneck Models Respect Localities?

Created by
  • Haebom

Author

Naveen Raman, Mateo Espinosa Zarlenga, Juyeon Heo, Mateja Jamnik

Outline

This paper analyzes the limitations of the concept-based explainability methodology. The concept-based explainability methodology uses a human-understandable intermediary to explain machine learning models, assuming that concept predictions help us understand the internal reasoning of the model. In this paper, we evaluate the validity of this assumption by analyzing whether concept predictors utilize “relevant” features for their predictions, that is, locality. Concept-based models that do not consider locality have poor explainability because concept predictions are based on features that are not apparent, making interpretation meaningless. In this paper, we present three metrics to evaluate locality and analyze them to complement theoretical results. Each metric captures the perturbation of different concepts and evaluates the impact of perturbation of “irrelevant” features on the concept predictor. We find that many concept-based models used in practice do not adhere to locality because concept predictors cannot always distinguish clearly distinct concepts, and we present proposals to alleviate this problem.

Takeaways, Limitations

Takeaways: We question the reliability of the concept-based explainability methodology and suggest new evaluation metrics and directions for improvement to enhance explainability by emphasizing the important aspect of locality. We present experimental results to evaluate whether the actual model complies with locality, and point out practical problems.
Limitations: Further research is needed to see if the three metrics presented are applicable to all types of concept-based models and datasets. The suggestions for improving locality are not presented as specific methodologies and may require further research. In addition, it should be noted that locality alone cannot fully guarantee explainability.
👍