Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

HERGC: Heterogeneous Experts Representation and Generative Completion for Multimodal Knowledge Graphs

Created by
  • Haebom

Author

Yongkang Xiao, Rui Zhang

Outline

This paper proposes HERGC, a novel multimodal knowledge graph completion (MMKGC) framework that leverages large-scale language models (LLMs) to address the incompleteness of multimodal knowledge graphs (MMKGs) that integrate diverse modalities (e.g., images and text). HERGC enriches and fuses multimodal information with heterogeneous expert representation searchers to retrieve a candidate set for each incomplete triple, and then accurately identifies the correct answer using a generative LLM predictor implemented via in-context learning or lightweight fine-tuning. Extensive experiments on three standard MMKG benchmarks demonstrate that HERGC outperforms existing methods.

Takeaways, Limitations

Takeaways:
A new approach to effectively utilize LLM at MMKGC
Improving MMKG completion performance by fusion of heterogeneous information and utilizing generative models.
Contributes to solving the closed world assumption and differential learning objective problem of the existing MMKGC method Limitations
Excellent performance verified in various MMKG benchmarks
Limitations:
High computational costs and potential resource consumption due to the use of LLM
Further research is needed on the explainability and reliability of LLM.
Potential performance bias for specific types of MMKG or modalities
May depend on the performance of the LLM used.
👍