Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

LUMA: A Benchmark Dataset for Learning from Uncertain and Multimodal Data

Created by
  • Haebom

Author

Grigor Bezirganyan, Sana Sellami, Laure Berti- Equille, S ebastien Fournier

Outline

LUMA is a new dataset designed to enhance the reliability of multimodal deep learning, which integrates diverse information sources (text, images, audio, and video) to improve decision making. It extends the existing CIFAR-10/100 dataset by adding audio and text data, and is specifically designed to allow for the infusion of various types and degrees of uncertainty, with the goal of learning from uncertain data. The audio data was extracted from three audio corpora, and the text data was generated using the Gemma-7B LLM. LUMA is available as a Python package that includes functions to generate multiple dataset variations by controlling data diversity, the amount of noise in each modality, and the addition of out-of-distribution samples. It also provides a baseline pretrained model and three uncertainty quantification methods: Monte-Carlo Dropout, Deep Ensemble, and Reliable Conflictive Multi-View Learning. This supports the development, evaluation, and benchmarking of reliable and robust multimodal deep learning methods.

Takeaways, Limitations

•
Takeaways:
◦
It facilitates the development of reliable multimodal models by providing multimodal datasets that can be injected with various types and levels of uncertainty.
◦
We provide uncertainty quantification methods and baseline models together to enhance researcher convenience.
◦
You can contribute to the design of more reliable and robust machine learning approaches for safety-critical applications.
◦
It is provided as a Python package, making it highly accessible and usable.
•
Limitations:
◦
Current uncertainty quantification methods may be limited.
◦
The size of the dataset may be relatively small compared to other large multimodal datasets.
◦
Additional validation of the quality and diversity of text data generated using Gemma-7B LLM may be required.
◦
It may not encompass all types of uncertainty.
👍
Made with Slashpage