This paper proposes HiLight, a hierarchical reinforcement learning framework for large-scale traffic signal control (TSC). To overcome the limitations of existing centralized and distributed reinforcement learning methods, HiLight consists of a high-level meta-policy that uses a Transformer-LSTM architecture to segment the traffic network into sub-regions and generate sub-goals, and low-level sub-policies that control individual intersections through global awareness. To enhance coordination between the meta-policy and sub-policies, an adversarial training mechanism is introduced. The meta-policy generates challenging yet informative sub-goals, and the sub-policies learn to excel at these goals. We evaluate HiLight on a large-scale Manhattan network encompassing synthetic and real-world benchmarks as well as diverse traffic conditions (e.g., peak-hour shifts, inclement weather, and holiday surges). We demonstrate significant performance gains in large-scale scenarios and competitive performance on standard benchmarks of various sizes.
