Electrocardiograms (ECGs) are essential tools for diagnosing cardiac disease, but diagnostic errors can occur due to noise or missing leads in standard 12-lead recordings. To address these issues, we propose TolerantECG, a foundation model for ECG signals that is robust to noise and can operate on any subset of the standard 12-lead ECG. TolerantECG training combines contrastive learning and self-supervised learning frameworks to simultaneously learn ECG signal representations, knowledge retrieval-based text report descriptions, and signals with damaged or missing leads.