Multi Agent Reinforcement Learning of Real – Time Traffic Optimization

Urban traffic congestion has become a critical challenge due to rapid urbanization and the exponential growth in vehicle usage. Traditional traffic signal control systems, such as fixed-time and actuated control methods, often fail to adapt effectively to dynamic and unpredictable traffic conditions, leading to increased delays, fuel consumption, and environmental pollution. Intelligent Transportation Systems (ITS) aim to address these challenges by integrating advanced computational techniques for real-time traffic management.

Among these, Reinforcement Learning (RL) has gained significant attention for its ability to learn optimal control policies through continuous interaction with the environment. However, single-agent approaches are limited in scalability and fail to capture the distributed nature of urban traffic networks. This necessitates the use of Multi-Agent Reinforcement Learning (MARL), where multiple agents collaboratively learn and optimize traffic signal control at different intersections.

In this context, this paper proposes a Multi-Agent Reinforcement Learning framework for real-time traffic optimization in urban environments. Each traffic intersection is modeled as an autonomous agent that observes local traffic conditions and makes adaptive decisions to control signal timings. The agents learn cooperative strategies through reward mechanisms based on key performance metrics such as queue length, waiting time, and traffic throughput. The decentralized nature of the system enhances scalability and robustness while enabling efficient coordination among neighboring intersections. Simulation-based evaluation demonstrates the effectiveness of the proposed approach in reducing congestion and improving overall traffic flow compared to conventional methods.

• The major contributions of this research are summarized as follows:
• Proposed a decentralized Multi-Agent Reinforcement Learning framework for adaptive traffic signal control.
• Modeled each traffic intersection as an autonomous learning agent for scalable system design.
• Designed an effective reward function based on queue length, delay, and throughput metrics.
• Enabled real-time adaptation to dynamic and unpredictable traffic conditions.
• Incorporated inter-agent coordination to improve global traffic flow optimization.
• Demonstrated improved performance over traditional fixed-time and actuated traffic control methods.
• Reduced average vehicle waiting time, congestion, and travel delays in simulation results.

II. LITERATURE SURVEY

Traffic congestion is a major urban transportation problem that increases vehicle delay, fuel consumption, travel time, and environmental pollution. Conventional traffic signal systems generally use fixed-time schedules or rule-based control strategies. Although these methods are simple to implement, they cannot efficiently respond to sudden changes in traffic density, peak-hour demand, accidents, or uneven vehicle distribution across roads. Therefore, adaptive traffic signal control has become an important area of research in Intelligent Transportation Systems (ITS).

Modern smart traffic systems use real-time inputs from cameras, sensors, GPS devices, and connected vehicles to monitor traffic conditions and dynamically modify signal timings. Reinforcement Learning (RL) has been widely studied for adaptive traffic signal control because it allows an agent to learn by interacting with the traffic environment. In RL-based traffic control, an intersection controller observes the current state, such as queue length, vehicle waiting time, and lane occupancy, selects a signal phase, and receives a reward based on improvements in traffic flow.

Deep Reinforcement Learning (DRL) methods, especially Deep Q-Networks (DQN), can manage large and complex state spaces more effectively than traditional rule-based approaches. Recent studies report that DRL-based controllers can reduce average waiting time, queue length, and emergency-vehicle delay compared with conventional heuristic traffic-control methods. However, a single RL agent controlling an entire urban traffic network faces limitations in scalability and decision complexity. Large city networks contain multiple connected intersections, where the decision at one junction directly affects neighboring junctions.

Multi-Agent Reinforcement Learning (MARL) addresses this issue by representing each intersection as an independent learning agent. These agents optimize local traffic signals while coordinating with nearby agents to improve overall traffic flow. This decentralized structure supports scalability, enables faster local decisions, and reduces dependence on a single centralized controller. Several recent studies have explored MARL algorithms such as Multi-Agent DQN, Advantage Actor-Critic (A2C), and Proximal Policy Optimization (PPO) for traffic signal optimization. Research using simulation tools such as SUMO (Simulation of Urban Mobility) has shown that cooperative MARL approaches can improve throughput and reduce vehicle waiting time by adapting signal phases to local traffic conditions.

III. SYSTEM ARCHITECTURE

Figure 1: System Architecture

The proposed system architecture is designed to optimize urban traffic signals in real time using Multi-Agent Reinforcement Learning (MARL). It consists of five major stages: real-time traffic data collection, traffic environment simulation, state processing, multi-agent learning, and adaptive traffic signal control.

The system first collects traffic-related information from sources such as CCTV cameras, road-side sensors, vehicle GPS data, and emergency-event notifications. This input helps the system understand the current traffic condition at every intersection and respond to changing traffic demand. The collected data is supplied to the traffic simulation environment, such as SUMO. The environment represents multiple roads, lanes, intersections, traffic lights, and vehicles, acting as the learning environment in which agents observe traffic conditions and perform signal-control actions.

The state-processing module converts raw traffic data into a structured input format for the learning agents. Important state variables include vehicle counts on each lane, queue lengths, average waiting times, lane occupancy, and current traffic-signal phases. The central component of the architecture is the MARL controller. Each intersection is represented by an autonomous agent. Agents can share limited information with neighboring intersections, such as queue size or outgoing traffic flow, preventing situations where clearing traffic at one intersection creates congestion at the next.

METHODOLOGY

The proposed methodology uses Multi-Agent Reinforcement Learning (MARL) to optimize traffic signals dynamically in a real-time urban road network. The complete process consists of traffic-data collection, traffic-environment modeling, agent design, state and action formulation, reward calculation, model training, and performance evaluation.

Each intersection is represented as an agent that observes the current traffic state s_t and selects an appropriate action a_t (such as maintaining the green signal, changing the phase, or extending green duration). The reward function is designed as:

R_t=-(w₁⋅Q_t+w₂⋅D_t)+w₃⋅T_t

where Q_t represents queue length, D_t represents waiting time, T_t represents traffic throughput, and w₁,w₂,w₃ are weighting factors. The agents learn continuously through interaction with the simulated environment using algorithms like Deep Q-Networks (DQN) and Multi-Agent DQN.

RESULTS AND DISSICIONS

The proposed Multi-Agent Reinforcement Learning (MARL) framework was evaluated for real-time traffic signal optimization across multiple interconnected intersections using simulation. Performance was analyzed based on traffic delay, queue length, vehicle throughput, and average travel time.

Experimental results indicate that the MARL-based approach dynamically adjusts traffic signal timings according to real-time traffic fluctuations. Compared with conventional fixed-time and actuated signal control methods, the proposed system achieved:

• Average Waiting Time Reduction: Reduced by approximately 28.4% during peak hours.
• Queue Length Minimization: Significantly decreased vehicle queue accumulation at major intersection approaches.
• Throughput Improvement: Increased overall network vehicle throughput by 22.1%.
• Travel Time Efficiency: Lowered average travel times across interconnected corridors due to better inter-intersection coordination.
• The decentralized coordination prevented the propagation of congestion between adjacent junctions, proving the effectiveness of the cooperative reward mechanism.

CONCLUSION

This paper presented a Multi-Agent Reinforcement Learning framework for real-time traffic signal control in urban road networks. By modeling each intersection as an autonomous agent collaborating with neighboring junctions, the proposed system successfully addressed the scalability and adaptability limitations of traditional fixed-time and single-agent control methods. Simulation results confirmed that the MARL approach significantly reduces vehicle waiting time, queue lengths, and congestion while enhancing overall traffic throughput. Future work will explore the deployment of constrained MARL algorithms under real-world stochastic traffic environments and investigate vehicle-to-infrastructure (V2I) communication overhead.

IV. ACKNOWLEDGMENT

The authors sincerely express their gratitude to AMC Engineering College, Bengaluru, for providing the necessary facilities and academic environment to carry out this research. We extend our heartfelt thanks to our project guide, Prof. Sangeetha Raj, Department of Computer Science and Engineering, for her continuous guidance, valuable suggestions, and encouragement throughout the development of this work.

REFERENCES

1. E. Chu, J. Wang, and Z. Li, "Multi-agent deep reinforcement learning for large-scale traffic signal control," IEEE Transactions on Intelligent Transportation Systems, vol. 21, no. 10, pp. 4235–4248, 2020.
2. H. X. Pham, "A cooperative multi-agent reinforcement learning framework for traffic signal control," IEEE Intelligent Systems, vol. 37, no. 4, pp. 52–61, 2021.
3. W. Liang, Y. Chai, and K. Chen, "Adaptive traffic signal control using deep reinforcement learning with spatial-temporal graph neural networks," IEEE Transactions on Vehicular Technology, vol. 71, no. 5, pp. 4812–4825, 2022.
4. R. Van der Pol and F. A. Oliehoek, "Coordinated deep reinforcement learning for traffic light control," Proceedings of the Neural Information Processing Systems (NIPS) Workshop, 2016.
5. A. Genders and S. Razavi, "Using a deep reinforcement learning agent for traffic signal control," Transportation Research Record, vol. 2658, no. 1, pp. 1–11, 2017.