Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning

Created by
  • Haebom

Author

Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Deli Zhao, Wenbing Huang, Tingyang Xu, Qifeng Bai, Yu Rong

ReasonMed: A large-scale inference-based language model for knowledge-intensive question answering in healthcare.

Outline

This paper explores the potential of inference-based large-scale language models in the knowledge-intensive medical question answering domain. To achieve this, we introduce ReasonMed, a large-scale medical inference dataset consisting of 370,000 high-quality examples. ReasonMed is derived from 1.75 million initial inference paths and built through a multi-agent generation, validation, and refinement process. Our results show that models trained using ReasonMed outperform previous state-of-the-art models on PubMedQA, demonstrating the effectiveness of a strategy that integrates detailed CoT inference with concise answer summaries.

Takeaways, Limitations

Takeaways:
A New Benchmark for Improving the Performance of Inference-Based LLM in the Healthcare Field
A model training strategy utilizing the ReasonMed dataset (combining CoT inference and answer summarization) is presented.
Achieved performance surpassing existing best-performing models in Sub-10B models.
Demonstrating the potential of model scaling
Limitations:
Specific Limitations is not specified in the paper (in general, dataset bias, model generalization ability, performance on other datasets, etc. may require further research)
👍