This paper proposes SCIZOR, a self-supervised learning-based data cleaning framework, to address the problem of poor quality in large-scale datasets in imitation learning, which trains robots to perform diverse behaviors. SCIZOR addresses two sources of poor data quality: suboptimal data (lack of task progression) and redundant patterns. Suboptimal data is removed using a self-supervised learning-based task progress predictor, and redundant data is removed using a deduplication module for the joint state-action representation. Experimental results demonstrate that SCIZOR achieves high-performance imitation learning policies even with limited data, achieving an average performance improvement of 15.4% across various benchmarks.