Project: #407 Alleviating Traffic Congestion: Developing and Evaluating Novel Multi-Agent Reinforcement Learning Traffic Light Coordination Techniques Progress Report - Reporting Period Ending: Sept. 30, 2022 Principal Investigator: Fei Fang Status: Active Start Date: July 1, 2022 End Date: June 30, 2023 Research Type: Advanced Grant Type: Research Grant Program: FAST Act - Mobility National (2016 - 2022) Grant Cycle: 2022 Mobility21 UTC Progress Report (Last Updated: Oct. 2, 2022, 7:51 p.m.) % Project Completed to Date: 20 % Grant Award Expended: 25 % Match Expended & Document: 25 USDOT Requirements Accomplishments The goal of this work is to design multi-agent reinforcement learning (MARL)-based algorithms for traffic signal control that would be deployable at scale. We particularly aim to enhance the efficiency of multi-agent coordination and communication in these MARL algorithms. A major prerequisite for deploying MARL in traffic signal control is understanding the practical considerations that would need to be incorporated into the algorithms by design. In our preliminary study, we reviewed some of these considerations, along with existing lines of work in MARL that have the potential to address them. We have identified four such considerations: (1) detection uncertainty; (2) reliability of communications between intersections; (3) compliance and interpretability; and (4) heterogeneous road users such as pedestrians and transit vehicles. Motivated by the first two considerations and our project goals, we are currently designing and experimenting with a MARL algorithm that uses graph neural networks to learn hierarchical, coordinated, and robust signal plans. At the same time, we are also collaborating with Econolite and PTV to deploy our work in the future. Impacts Our review paper has filled a gap in the literature on MARL for traffic signal control, which to date has focused more on algorithmic contributions than practical considerations. It takes a step towards building a common understanding of requirements and desiderata between experts in transportation and artificial intelligence. The paper was well-received by the audience of artificial intelligence experts at the 12th Workshop on Agents in Traffic and Transportation (ATT) in July. Meanwhile, through our industry collaborations, we have shared the concept of MARL as a new approach to adaptive traffic signal control with our partners, including Econolite, PTV, and the city of Strongsville, Ohio, who are very excited about the opportunities this technology can provide. Other N/A Outcomes New Partners We have been discussing with stakeholders in the city of Strongsville, Ohio, the possibility of testing our algorithm there. Issues It took longer for us to get access to PTV, a software on which we can build our simulator. The issues are resolved in the past two weeks. This unexpected delay would potentially lead to a delay in building our simulator.