[공지사항]을 빙자한 안부와 근황 
Show more

Daily Arxiv

This is a page that curates AI-related papers published worldwide.
All content here is summarized using Google Gemini and operated on a non-profit basis.
Copyright for each paper belongs to the authors and their institutions; please make sure to credit the source when sharing.

OrthoInsight: Rib Fracture Diagnosis and Report Generation Based on Multi-Modal Large Models

Created by
  • Haebom

Author

Ningyong Wu, Jinzhi Wang, Wenhong Zhao, Chenzhan Yu, Zhigang Xiu, Duwei Dai

Outline

In this paper, we propose OrthoInsight, a multimodal deep learning framework for thoracic fracture diagnosis. OrthoInsight integrates the YOLOv9 model for fracture detection, a medical knowledge graph for clinical context retrieval, and a fine-tuned LLaVA language model for diagnostic report generation. It combines visual features of CT images with text data from experts to provide clinically useful results. In an evaluation using 28,675 annotated CT images and expert reports, it achieves high performance (average score of 4.28) in diagnostic accuracy, content completeness, logical consistency, and clinical guideline value, outperforming models such as GPT-4 and Claude-3. This study demonstrates the potential of multimodal learning to transform medical image analysis and provide effective support to radiologists.

Takeaways, Limitations

Takeaways:
We present the feasibility of automating the diagnosis of thoracic fractures using multimodal deep learning.
We demonstrate superior performance of OrthoInsight by integrating YOLOv9, medical knowledge graph, and LLaVA.
Introducing a new paradigm for medical image analysis and support for radiologists.
It achieved higher performance than existing models such as GPT-4 and Claude-3.
Limitations:
The paper lacks specific references to Limitations or future research directions.
Additional validation of the generalizability of the dataset used may be needed.
Further research and validation are needed for clinical application.
👍