To overcome the limitations of the existing 7-Point Checklist (7PCL), this paper proposes a novel diagnostic framework that integrates the Clinical Knowledge-Based Topological Graph (CKTG) and the Gradient Diagnosis Strategy (GD-DDW) with a data-driven weighting system. CKTG captures internal and external relationships between 7PCL attributes, while GD-DDW prioritizes visual observation, mimicking the diagnostic process of dermatologists. Furthermore, we introduce a multimodal feature extraction method utilizing a dual-attention mechanism to enhance feature extraction through cross-modal interaction and unimodal collaboration, and integrate meta-information to uncover the interactions between clinical data and image features. Evaluation results using the EDRA dataset demonstrated excellent performance in melanoma detection and feature prediction, achieving an average AUC of 88.6%. This integrated system provides clinicians with a data-driven benchmark, significantly improving the accuracy of melanoma diagnosis.