Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Can Large Multimodal Models Understand Agricultural Scenes? Benchmarking with AgroMind

Created by
  • Haebom

Author

Qingmei Li, Yang Zhang, Zurong Mai, Yuhang Chen, Shuohong Lou, Henglian Huang, Jiarui Zhang, Zhiwei Zhang, Yibin Wen, Weijia Li, Haohuan Fu, Jianxi Huang, Juepeng Zheng

Outline

This paper presents AgroMind, a comprehensive benchmark specialized for agricultural remote sensing. To overcome the limitations of existing benchmarks, which include limited dataset diversity and oversimplified task design, we integrated eight public datasets and one private farmland dataset to build a high-quality evaluation set containing 27,247 QA pairs and 19,615 images. AgroMind covers 13 task types (ranging from crop identification and health monitoring to environmental analysis) across four task dimensions: spatial perception, object understanding, scene understanding, and scene inference. Evaluating 20 open-source LMMs and four closed-source models with AgroMind reveals significant performance differences, particularly in spatial inference and fine-grained recognition, with some leading LMMs outperforming human performance. AgroMind establishes a standardized evaluation framework for agricultural remote sensing, exposing the domain limitations of LMMs and highlighting important challenges for future research. Data and code are available at https://rssysu.github.io/AgroMind/ .

Takeaways, Limitations

Takeaways:
AgroMind provides comprehensive and standardized benchmarks for agricultural remote sensing.
Establishing a foundation for objectively evaluating and improving LMM performance
Limitations of LMM's domain knowledge and future research directions (especially spatial reasoning and fine-grained perception)
We demonstrate that some LMMs can outperform humans, confirming the potential of LMMs for further development.
Limitations:
Further research is needed to understand the scope and diversity of the AgroMind dataset.
More diverse LMM models need to be evaluated.
A deeper analysis of the causes of performance deviations for specific task types is needed.
👍