As deep neural networks are utilized in various applications, network checkpoints are being shared and distributed to facilitate the development process. This paper addresses the threat of stegomalware, which hides malware in deep neural network checkpoints with minimal impact on network accuracy. Despite being a critical security issue, this has received little attention from deep learning researchers and security professionals. In this paper, we propose the first effective countermeasure against this attack. Specifically, we demonstrate that state-of-the-art stegomalware can be effectively neutralized by shuffling the column order of the weight and bias matrices, or the channel order of the convolutional layers. This effectively corrupts payloads embedded using state-of-the-art steganographic methods without compromising network accuracy, significantly outperforming competing methods. We also advocate ongoing research into methods to circumvent this defense, additional defenses, and the security of machine learning systems.