Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models

Created by
  • Haebom

Author

Anindya Bijoy Das, Shibbir Ahmed, Shahnewaz Karim Sakib

Outline

This paper investigates the effectiveness of an open-source large-scale language model (LLM) for extracting key events (reasons for admission, critical in-hospital events, and important follow-up actions) from medical record summaries, particularly discharge reports. To rigorously evaluate the accuracy and fidelity of LLM, we conduct comprehensive simulations and analyze the prevalence of hallucinations during the summarization process. While LLMs such as Qwen2.5 and DeepSeek-v2 successfully capture reasons for admission and events occurring during hospitalization, they are inconsistent in identifying follow-up recommendations.

Takeaways, Limitations

Takeaways: Demonstrated the effectiveness of open-source LLM in extracting reasons for admission and major hospital events from medical record summaries. This suggests the potential for developing an LLM-based automated summarization system.
Limitations: LLMs lack consistency in identifying follow-up recommendations. The presence of hallucinations compromises the reliability of LLM summaries. LLMs pose challenges in utilizing comprehensive medical record summaries.
👍