In this paper, we propose ConTextual, a novel framework for extracting important information from unstructured clinical data and utilizing it for patient care decision making. To address the problem that existing studies either treat all tokens equally or rely on heuristic-based filters that overlook important clinical information, ConTextual integrates a context-preserving token filtering method with a domain-specific knowledge graph (KG). By preserving context-specific important tokens and enriching them with structured knowledge, it improves both linguistic consistency and clinical accuracy. Experimental results on two public benchmark datasets show that ConTextual consistently outperforms other baseline models. It highlights the complementary roles of token-level filtering and structured retrieval, and provides a scalable solution for improving the accuracy of clinical text generation.
