Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

Are Vision Foundation Models Ready for Out-of-the-Box Medical Image Registration?

Created by
  • Haebom

Author

Hanxue Gu, Yaqian Chen, Nicholas Konz, Qihang Li, and Maciej A. Mazurowski

Outline

This paper evaluates the performance of breast MRI image registration using a pre-trained base model on a large-scale image dataset. Unlike previous studies targeting rigid bodies or relatively simple structures (e.g., the brain, abdominal organs), we focused on image registration of highly deformable breast tissue. Using five pre-trained encoders—Dino-v2, SAM, MedSAM, SSLSAM, and MedCLIP—we performed breast image registration under various conditions, including year, sequence, modality, and lesion presence, and compared their performance. Our results show that base model-based algorithms, such as SAM, outperform existing algorithms in overall breast alignment performance, but struggle with aligning the fine structures of fibrous tissue. Furthermore, we found that additional pre-training or fine-tuning with medical or breast-specific images did not improve performance and, in some cases, actually reduced it. The code is publicly available.

Takeaways, Limitations

Takeaways:
We demonstrate that model-based image registration algorithms (especially SAM) outperform existing algorithms in overall alignment performance for breast MRI image registration.
The strengths of the base model become more evident when domain changes are significant.
Open code facilitates further research and development.
Limitations:
The base model has difficulty in accurately aligning microstructures such as fiber-like structures.
Additional pre-training or fine-tuning using medical or breast-specific images may not improve performance or may even degrade it.
Further research is needed on the impact of domain-specific training on image registration.
Further research is needed on strategies to improve both overall alignment and microstructural accuracy.
👍