Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Aligning Machine and Human Visual Representations across Abstraction Levels

Created by
  • Haebom

Author

Lukas Muttenthaler, Klaus Greff, Frieda Born, Bernhard Spitzer, Simon Kornblith, Michael C. Mozer, Klaus-Robert Muller , Thomas Unterthiner, Andrew K. Lampinen

Outline

This paper highlights that while deep neural networks have demonstrated success as neural representation models for human behavior and visual tasks, they learn fundamentally differently from human learning and fail to achieve robust generalization capabilities. A key discrepancy lies in the fact that human conceptual knowledge is hierarchically organized, ranging from fine-grained to macroscopic, while model representations fail to accurately capture all these levels of abstraction. To address this, we train a teacher model to mimic human judgment, then fine-tune the representations of a pre-trained, state-of-the-art vision-based model to transfer a human-aligned structure. The result is a human-aligned model that more accurately approximates human behavior and uncertainty across a variety of similarity tasks and improves generalization and distributional robustness across a variety of machine learning tasks. In conclusion, we demonstrate that adding human knowledge to neural networks yields optimal representations that enhance both human cognitive judgment and practical application, leading to more robust, interpretable, and human-aligned artificial intelligence systems.

Takeaways, Limitations

Takeaways:
A novel method for integrating human hierarchical conceptual knowledge structures into models is presented.
Human-aligned models more accurately reflect human behavior and uncertainty across a variety of tasks.
Improving generalization and distributional robustness of human-aligned models.
Presents the potential for developing more robust, interpretable, and human-aligned AI systems.
Limitations:
The generalizability and representativeness of human judgment data used to train teacher models needs to be reviewed.
The generalization performance of the proposed method needs to be evaluated on other visual tasks and other types of data.
Further research is needed on the limitations and ways to improve the ability to fully capture the hierarchical structure of human knowledge.
👍