This paper proposes a novel deep learning-based denoising framework to address noise and artifact issues in ultra-low-dose computed tomography (uLDCT) images. Specifically, to address the spatial mismatch between uLDCT and non-dose computed tomography (NDCT), we introduce an Image Purification (IP) strategy to generate structurally aligned uLDCT-NDCT pairs. Based on this, we develop a Frequency-domain Flow Matching (FFM) model to enhance anatomical preservation. Experiments using real clinical data demonstrate that the combination of the proposed IP strategy and the FFM model outperform existing denoising models.