[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Lessons from the TREC Plain Language Adaptation of Biomedical Abstracts (PLABA) track

Created by
  • Haebom

Author

Brian Ondov, William Xia, Kush Attal, Ishita Unde, Jerry He, Hoa Dang, Ian Soboroff, Dina Demner-Fushman

Outline

This paper presents results from the Plain Language Adaptation of Biomedical Abstracts (PLABA) track at the Text Retrieval Conferences in 2023 and 2024. The PLABA track focuses on converting technical abstracts of medical papers into plain language that is easy for the general public to understand. A variety of models ranging from multilayer perceptrons to pre-trained giant language models (LLMs) are evaluated on two tasks: Task 1: rewriting the entire abstract and Task 2: identifying and replacing difficult terms. In Task 1, top-tier models achieve expert-level accuracy and completeness, but lack conciseness and clarity, and automated evaluation metrics have poor correlation with manual evaluation. In Task 2, LLM-based systems struggle with identifying difficult terms and classifying replacement methods, but perform well in generating replacement terms in terms of accuracy, completeness, and conciseness.

Takeaways, Limitations

Takeaways:
We demonstrate the potential of leveraging large language models to translate expert medical papers into language accessible to the general public.
The LLM-based system showed significant performance in the medical terminology substitution task.
It suggests the possibility of developing a medical information transformation model with expert-level accuracy and completeness.
Limitations:
The correlation between automated evaluation metrics and manual evaluations is low, requiring the development of improved automated evaluation tools.
Even the higher-end models still have room for improvement when it comes to simplicity and clarity.
They had difficulty identifying difficult terms and selecting appropriate substitutes.
👍