Energy-Aware Traffic Signal Optimization for Sustainable Urban Networks Using Valid Action Policies

Optimizing signal control across urban networks is crucial for reducing energy consumption and improving traffic efficiency. Advances in Vehicle-to-Everything (V2X) technologies have enabled deep reinforcement learning (DRL) to demonstrate significant promise in traffic signal control. However, many existing studies focus primarily on agent performance while neglecting practical operational constraints, which can lead to fairness issues in traffic control. Moreover, prior work often treats energy optimization as a by-product of efficiency. To address these gaps, this study proposes a multi-agent DRL framework to mitigate peak-hour congestion and reduce excessive emissions in urban networks. Building on this framework, a variant of Independent Proximal Policy Optimization (IPPO), termed Global State Independent PPO (GS-IPPO), is developed to enhance IPPO's coordination capability. To further improve practical applicability and fairness, this study incorporates constraints imposed by the Ring-and-Barrier structure. It implements real-time masking of infeasible or unreasonable actions, allowing for phase transitions that better align with real-world traffic requirements. In addition, the proposed framework explicitly performs active energy optimization. A real-world network, including a T-intersection, is modeled in SUMO in the Shinan District, Qingdao, China, using demand data estimated from AMap. The experimental results demonstrate that GS-IPPO consistently outperforms both conventional and learning-based baselines in vehicle delay and energy consumption. Specifically, it reduces average delay by 62% and energy consumption by 46% compared with fixed-time control, and achieves reductions of 11% and 8.7% relative to Vehicle-Actuated Control. Even against advanced multi-agent RL methods such as IPPO and Multi-Agent Proximal Policy Optimization (MAPPO), GS-IPPO yields additional improvements of 7.7% in delay and 4.5% in energy consumption, underscoring its clear advantage in traffic signal control.