Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Impact of Stickers on Multimodal Sentiment and Intent in Social Media: A New Task, Dataset and Baseline

Created by
  • Haebom

Author

Yuanchen Shi, Biao Ma, Longyin Zhang, Fang Kong

Outline

In this paper, we present a new task, Multimodal chat Sentiment Analysis and Intent Recognition involving Stickers (MSAIRS), to analyze the influence of stickers, which are increasingly used to express emotions and intentions in social media. We introduce a new multimodal dataset containing Chinese chat transcripts and stickers, which includes various stickers with different stickers for the same text, different contexts for the same sticker, and different texts for the same image, to better understand the influence of stickers on chat sentiments and intentions. In addition, we propose an effective multimodal joint model, MMSAIR, featuring discriminant vector construction and cascading attention mechanism, which demonstrates improved accuracy through mutual reinforcement of emotions and intentions. Experimental results show that MMSAIR outperforms existing models and advanced MLLMs, demonstrating the challenge and uniqueness of sticker interpretation in social media. The dataset and code are open-sourced on GitHub.

Takeaways, Limitations

Takeaways:
We highlight the importance of stickers in social media sentiment analysis and intent recognition research and suggest new research directions.
We propose a multi-modal dataset and model that considers different aspects of stickers (text, image, and context).
We experimentally demonstrate the effectiveness of joint modeling that takes into account the interdependence of emotions and intentions.
We contribute to the development of sticker-based emotion and intention analysis technology by presenting the MMSAIR model that outperforms existing models.
Publicly available datasets and code provide a foundation for further research.
Limitations:
The current dataset is limited to Chinese social media data. Further research is needed to determine generalizability to other languages and cultures.
Additional research may be needed to interpret the visual meaning of stickers. Current models may not fully capture the visual features of images.
It may not cover all the different types of stickers and complex social media situations. A more diverse and richer dataset may be needed.
👍