This paper investigates the effectiveness of open-source large-scale language models (LLMs) for extracting key events (reasons for admission, major in-hospital events, and important follow-up measures) from medical summaries, specifically discharge reports. We also evaluate the incidence of hallucinations, which can affect the accuracy and reliability of LLMs. Experiments using LLMs such as Qwen2.5 and DeepSeek-v2 demonstrate excellent performance in extracting reasons for admission and events occurring during hospitalization, but show inconsistencies in identifying follow-up recommendations. This highlights the challenges of leveraging LLMs for comprehensive summarization.
