In this paper, we propose MedTok, which improves the tokenization of medical data used in the base model trained on patient electronic health records (EHRs). While existing tokenization methods treat medical codes as simple text tokens, MedTok considers the text description of a medical code, its hierarchical position, and its relationships with other codes (e.g., disease co-occurrence, drug therapy associations). It processes the text and relational structure using a language model encoder and a graph encoder, and quantizes it into a unified token space to preserve modality features and cross-modality information. In various experiments (prediction, diagnosis classification, drug recommendation, and risk stratification) using MIMIC-III, MIMIC-IV, and EHRShot datasets, it improves AUPRC compared to existing tokenization methods, and shows great performance in drug recommendation in particular. In addition, we applied MedTok to a medical QA system and confirmed its performance improvement.
