This paper proposes the Shielded Multi-Agent Reinforcement Learning (SMARL) framework, which extends Probabilistic Logic Shields (PLS), which guarantees safety in single-agent reinforcement learning, to multi-agent environments. SMARL introduces a novel Probabilistic Logic Temporal Difference (PLTD) update method that directly integrates probabilistic constraints into the value update process, and a probabilistic logic policy gradient method that provides formal safety guarantees for MARL. We evaluate SMARL on various n-player game theory benchmarks with symmetric and asymmetric constraints, demonstrating that it reduces constraint violations and significantly improves cooperation compared to existing methods. This suggests that SMARL can be established as an effective mechanism for secure and socially harmonious multi-agent systems.