This paper addresses source separation, a fundamental challenge in speech, music, and audio processing. Specifically, we propose a training-time and inference-time scalable discriminative source separation (TISDiSS) framework to address the issue of large-scale network dependency, which increases training and deployment costs. TISDiSS integrates early-split multi-loss supervision, shared-parameter design, and dynamic inference repetitions to enable flexible speed-performance tradeoffs by adjusting the inference depth without additional model retraining. The proposed method is useful for low-latency applications and achieves state-of-the-art performance on standard speech separation benchmarks while reducing the number of parameters.
