Daily Arxiv

This page organizes papers related to artificial intelligence published around the world.
This page is summarized using Google Gemini and is operated on a non-profit basis.
The copyright of the paper belongs to the author and the relevant institution. When sharing, simply cite the source.

Generative Medical Event Models Improve with Scale

Created by
  • Haebom

Author

Shane Waxler, Paul Blazek, Davis White, Daniel Sneider, Kevin Chung, Mani Nagarathnam, Patrick Williams, Hank Voeller, Karen Wong, Matthew Swanhorst, Sheng Zhang, Naoto Usuyama, Cliff Wong, Tristan Naumann, Hoifung Poon, Andrew Loza, Daniella Meeker, Seth Hain, Rahul Shah

Outline

This paper presents the Comet model, a decoder-only transformer-based medical event generation model trained on 151 billion tokens of medical events from 118 million patient data, using the large-scale healthcare dataset Epic Cosmos, containing 16.3 billion medical events. The Comet model simulates a patient's health progression by predicting the next medical event based on the patient's past medical history. Experimental results on 78 real-world healthcare tasks (e.g., diagnosis prediction, disease prognosis prediction, healthcare management, etc.) show that the Comet model performs similarly or better than task-specific supervised learning models, achieving these results without any special fine-tuning or small-sample training. This study reveals a power-law scaling relationship between computing resources, the number of tokens, and model size, and presents a methodology for pretraining computationally efficient models with up to 1 billion parameters.

Takeaways, Limitations

•
Takeaways:
◦
We demonstrated the utility of a medical event generation model leveraging large-scale medical data.
◦
We present a generalizable model that demonstrates excellent performance for a variety of medical tasks.
◦
It achieves high performance without fine-tuning or small-sample learning, increasing practicality.
◦
We observed improved predictive power as the model and pre-training scale increased.
◦
We suggest future research directions by presenting a scaling relationship between computing resources, number of tokens, and model size.
•
Limitations:
◦
Detailed descriptions of the characteristics of the Epic Cosmos dataset (data sources, patient demographic diversity, etc.) may be lacking.
◦
There may be a lack of discussion about the interpretability of the model.
◦
Further validation of its applicability and safety in real-world clinical settings is needed.
◦
Sufficient consideration must be given to data privacy and ethical issues.
👍
Made with Slashpage