This paper presents VolleyBots, a robotic sports testbed where multiple drones cooperate and compete in a volleyball game. VolleyBots is a platform that integrates three features: competitive and cooperative gameplay, turn-based interaction structure, and agile 3D maneuvering. Drones are challenged to coordinate with their teammates and anticipate and respond to the tactics of the opposing team. Turn-based interaction requires precise timing, accurate state prediction, and management of long-term temporal dependencies, while agile 3D maneuvering requires rapid acceleration, sharp turns, and accurate 3D positioning despite the lack of quadrotor drive. In this paper, we present a comprehensive set of tasks ranging from single-drone training to multi-drone cooperation and competition tasks, and a baseline evaluation of representative multi-agent reinforcement learning (MARL) and game theory algorithms. Simulation results show that on-policy reinforcement learning (RL) methods outperform off-policy methods in single-agent tasks, but both methods struggle in complex tasks that combine motor control and strategic play. We also design a hierarchical policy that achieves a win rate of 69.5% over the strongest baseline in a 3v3 task, highlighting its potential as an effective solution for addressing complex interactions between low-level control and high-level strategy.
