This paper addresses the issue of variability in Whole-Slide Images (WSI) due to differences in digital scanners, which is crucial for ensuring reliable model performance across domains in computational pathology. Specifically, we emphasize the importance of scanner generalization, which ensures model independence from scanner dependence in real-world settings where scanning devices may vary across institutions and hospitals. Unlike previous studies that primarily focus on standard domain generalization settings, this paper presents SCORPION, a novel dataset explicitly designed to evaluate model reliability under scanner variability. SCORPION comprises 2,400 spatially aligned patches of 480 tissue samples scanned by five scanners, isolating between-scanner variability to control for differences in tissue composition while rigorously assessing model consistency. Furthermore, we propose SimCons, a flexible framework that explicitly addresses the scanner generalization problem by combining augmentation-based domain generalization techniques with consistency loss. Experimental results demonstrate that SimCons improves model consistency across scanners without compromising task-specific performance. By releasing the SCORPION dataset and the SimCons framework, we provide the research community with a valuable resource for evaluating and improving model consistency across a variety of scanners and set a new standard for reliability testing.
