[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

ContextQFormer: A New Context Modeling Method for Multi-Turn Multi-Modal Conversations

Created by
  • Haebom

Author

Yiming Lei, Zhizheng Yang, Zeming Liu, Haitao Leng, Shaoguo Liu, Tingting Gao, Qingjie Liu, Yunhong Wang

Outline

This paper addresses the issue that existing open source models have weak multi-turn interaction capabilities, especially in long contexts, despite their zero-shot capabilities and powerful image understanding capabilities. To address these issues, we propose a context modeling module called ContextQFormer that enhances the representation of contextual information, and announce the construction and release of a new dataset, TMDialog, for multi-turn multi-modal dialogue research. TMDialog supports multi-turn multi-modal dialogue research, including longer conversations than existing datasets. In experiments using TMDialog, ContextQFormer shows 2-4% better performance than existing models.

Takeaways, Limitations

•
Takeaways:
◦
A new context modeling module (ContextQFormer) is presented to contribute to improving the performance of multi-turn multi-modal conversation models with long contexts.
◦
We expect to stimulate future research through the release of a new large-scale dataset (TMDialog) for multi-time multi-modal dialogue research.
◦
ContextQFormer shows performance improvement over existing models (2%-4%)
•
Limitations:
◦
Lack of detailed information about the specific composition and size of the TMDialog dataset, and data quality.
◦
Performance improvements for ContextQFormer may be relatively small, 2-4%
◦
Lack of comparative analysis with other multimodal conversation models requires further verification of generalizability.
👍
Made with Slashpage