In this paper, we propose a novel method, Simultaneous MRMP Diffusion (SMD), to solve the multi-robot motion planning (MRMP) problem using a diffusion model. SMD generates collision-free and kinematically plausible trajectories by leveraging constrained optimization, which incorporates important constraints such as collision avoidance and kinematic plausibility into the diffusion sampling process. In addition, we present a comprehensive MRMP benchmark to evaluate trajectory planning algorithms under scenarios with various robot densities, obstacle complexities, and motion constraints. Experimental results show that SMD achieves higher success rate and efficiency than existing methods and other learning-based motion planning algorithms in complex multi-robot environments.