This paper presents a self-supervised learning-based multimodal framework for integrating various clinical data modalities to improve stroke risk prediction. It combines 3D brain images, clinical data, and image-derived features to improve pre-stroke risk prediction. It utilizes an unlabeled dataset (UK Biobank) to capture complementary and synergistic information between image and tabular data modalities. Based on the contrastive learning framework, it combines contrastive word-image pre-training and image-table data matching modules to align multimodal data representations into a shared latent space. The results are compared and evaluated with the existing best-performing unimodal and multimodal methods under various model settings (fixed and trainable), and the ROC-AUC is improved by 2.6% (2.6%) and the balanced accuracy by 3.3% (5.6%) over the self-supervised learning tabular data (image) method, and the balanced accuracy is improved by 7.6% over the best-performing multimodal supervised learning model. We demonstrate that improved integration of tabular and image data through interpretable tools provides richer and more aligned embeddings, while Gradient-weighted Class Activation Mapping heatmaps reveal activation of brain regions associated with brain aging, stroke risk, and clinical outcomes.
