This paper presents a Multi-Fidelity, Model Fusion strategy to address the performance variability issue of Multiple Instance Learning (MIL) for classification of Whole Slide Images (WSIs) in digital pathology. Due to the high resolution and large size of WSIs, it is difficult to apply deep learning models, and the MIL method solves this problem by using global labels per slide, but it has a large performance variability due to factors such as weight initialization, batch order, and learning rate. In this paper, we propose a method to reduce this variability by training multiple models for several epochs and averaging the most stable and promising models based on the validation scores. The effectiveness of the proposed method is verified through more than 2000 experiments using two datasets, three initialization strategies, and five MIL methods.