Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Advancing Stroke Risk Prediction Using a Multi-modal Foundation Model

Created by
  • Haebom

Author

Camille Delgrange, Olga Demler, Samia Mora, Bjoern Menze, Ezequiel de la Rosa, Neda Davoudi

Outline

This paper presents a self-supervised learning-based multimodal framework for integrating various clinical data modalities to improve stroke risk prediction. It combines 3D brain images, clinical data, and image-derived features to improve pre-stroke risk prediction. It utilizes an unlabeled dataset (UK Biobank) to capture complementary and synergistic information between image and tabular data modalities. Based on the contrastive learning framework, it combines contrastive word-image pre-training and image-table data matching modules to align multimodal data representations into a shared latent space. The results are compared and evaluated with the existing best-performing unimodal and multimodal methods under various model settings (fixed and trainable), and the ROC-AUC is improved by 2.6% (2.6%) and the balanced accuracy by 3.3% (5.6%) over the self-supervised learning tabular data (image) method, and the balanced accuracy is improved by 7.6% over the best-performing multimodal supervised learning model. We demonstrate that improved integration of tabular and image data through interpretable tools provides richer and more aligned embeddings, while Gradient-weighted Class Activation Mapping heatmaps reveal activation of brain regions associated with brain aging, stroke risk, and clinical outcomes.

Takeaways, Limitations

Takeaways:
We improved stroke risk prediction performance through a multimodal framework based on self-supervised learning.
It outperforms existing best-performing single-modal and multi-modal methods.
Provides insight into the model's prediction results through interpretable tools.
Provides a powerful foundation for integrating diverse data modalities.
Limitations:
This study relies on the UK Biobank dataset, and generalization performance to other datasets requires further study.
Due to the nature of self-supervised learning, performance may be affected by the quality of unlabeled data.
Further research is needed to explore the interpretability of the model.
Generalization performance estimates for specific populations may be lacking.
👍