This paper proposes a novel Test-Time Adaptation (TTA) framework to address the performance degradation associated with out-of-distribution samples in image-to-image transformation of medical images. This framework quantifies the degree of domain shift through a Reconstruction Module and introduces a Dynamic Adaptation Block that dynamically adjusts the internal features of a pre-trained transformation model to adapt to out-of-distribution samples. Adaptation is not applied to in-distribution samples, preventing performance degradation. TTA demonstrates performance improvements over existing methods and baseline models without TTA in two medical image transformation tasks: low-dose CT denoising and T1-to-T2 MRI conversion. We highlight the limitations of existing state-of-the-art methods, which apply adaptation equally to both in- and out-of-distribution samples, and demonstrate that sample-specific dynamic adaptation is a promising approach to enhance model robustness in real-world environments.