In this paper, we present IKDiffuser, a diffusion-based model for solving inverse kinematics (IK) problems for multi-robot arm systems. While existing IK solvers suffer from the problems of slowness, failure-proneness, and lack of solution diversity due to complex self-collisions, coupled joints, and high-dimensional redundancy of degrees of freedom, IKDiffuser learns joint distributions over the configuration space, enabling smooth generalization to multi-robot arm systems with diverse architectures. In addition, it can integrate additional objectives during inference without retraining, providing diversity and adaptability to task-specific requirements. Experimental results on six different multi-robot arm systems demonstrate that IKDiffuser achieves superior solution accuracy, precision, diversity, and computational efficiency compared to existing solvers.