This page organizes papers related to artificial intelligence published around the world. This page is summarized using Google Gemini and is operated on a non-profit basis. The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.
ReasonMed: A 370K Multi-Agent Generated Dataset for Advancing Medical Reasoning
Created by
Haebom
Author
Yu Sun, Xingyu Qian, Weiwen Xu, Hao Zhang, Chenghao Xiao, Long Li, Deli Zhao, Wenbing Huang, Tingyang Xu, Qifeng Bai, Yu Rong
ReasonMed: A large-scale inference-based language model for knowledge-intensive question answering in healthcare.
Outline
This paper explores the potential of inference-based large-scale language models in the knowledge-intensive medical question answering domain. To achieve this, we introduce ReasonMed, a large-scale medical inference dataset consisting of 370,000 high-quality examples. ReasonMed is derived from 1.75 million initial inference paths and built through a multi-agent generation, validation, and refinement process. Our results show that models trained using ReasonMed outperform previous state-of-the-art models on PubMedQA, demonstrating the effectiveness of a strategy that integrates detailed CoT inference with concise answer summaries.
Takeaways, Limitations
•
Takeaways:
◦
A New Benchmark for Improving the Performance of Inference-Based LLM in the Healthcare Field
◦
A model training strategy utilizing the ReasonMed dataset (combining CoT inference and answer summarization) is presented.
◦
Achieved performance surpassing existing best-performing models in Sub-10B models.
◦
Demonstrating the potential of model scaling
•
Limitations:
◦
Specific Limitations is not specified in the paper (in general, dataset bias, model generalization ability, performance on other datasets, etc. may require further research)