Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

Format Inertia: A Failure Mechanism of LLMs in Medical Pre-Consultation

Created by
  • Haebom

Author

Seungseop Lim, Gibaeg Kim, Wooseok Han, Jean Seo, Hyunkyung Lee, Jaehyo Yoo, Eunho Yang

Outline

Advances in large-scale language models (LLMs) have led to significant improvements in various service areas, such as chatbots and medical pre-consultation applications. Supervised Fine-Tuning (SFT) is the most common method for adapting LLMs to multi-turn dialogue generation in the medical field. However, datasets for SFTs in tasks such as medical pre-consultation typically have an imbalanced turn distribution. Training on such data introduces a novel failure mechanism, known as "Format Inertia," which causes the model to generate repetitive, formally correct, but diagnostically uninformative questions in long medical conversations. To mitigate this failure mechanism, we adopted a simple, data-driven method to rebalance the turn distribution of the training dataset. Experimental results demonstrate that our method substantially mitigates Format Inertia in medical pre-consultation.

Takeaways, Limitations

Takeaways:
We identified a new failure mechanism called Format Inertia.
We demonstrate that format inertia can be effectively mitigated using a data-centric approach.
It can contribute to improving the performance of multi-turn dialogue systems such as medical pre-consultation.
Limitations:
Further research is needed to determine the generalizability of the proposed method.
No consideration was given to other failure mechanisms.
There is a need to explore other ways to improve beyond simple data rebalancing.
👍