Reinforcement Learning for Traffic Signal Control

The aim of this repository is to offering comprehensive dataset, simulator, relevant papers and survey to anyone who may wish to start investigation or evaluate a new algorithm.

Table of contents

• Key paper list
• Open Datasets
• Traffic Simulator
• A comprehensive survey

Key paper list

Method	Paper	Published	Notes	Code	Demo video	Poster
MPLight	Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control	AAAI'2020	A combination of PressLight and FRAP	-	Demo	-
CoLight	CoLight: Learning Network-level Cooperation for Traffic Signal Control	CIKM'19	Attention-based coordination	Code	N/A	poster
PressLight	PressLight: Learning Max Pressure Control to Coordinate Traffic Signals in Arterial Network	KDD'19	Pressure-based coordination	Code	Demo	poster
FRAP	Learning Phase Competition for Traffic Signal Control	CIKM'19	Our most powerful single intersectiton control model	Code	N/A	poster
MetaLight	MetaLight: Value-based Meta-reinforcement Learning for Traffic Signal Control	AAAI'2020	Meta-RL for traffic signal control]	Code	-	-
DemoLight	Learning Traffic Signal Control from Demonstrations	CIKM'19	Learn from expert demonstrations	Code	N/A	poster
IntelliLight	IntelliLight: A Reinforcement Learning Approach for Intelligent Traffic Light Control	KDD'18	First try on RL signal control. The base of all the methods	N/A	Demo	poster
CityFlow	CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario	WWW'19 Demo	Simulator	Code	Demo	N/A

Open datasets

We provide different traffic datasets, each includes both road network (roadnet.json) and traffic flow file (flow.json), whose formats are defined in Roadnet File Format and Flow File Format respectively.

*All methods are measured in Average Travel Time (in seconds) under CityFlow simulator.

#	Dataset name	Number of Intersections	Time Span (Seconds)	Description	Referred result*	Referred method
1	hangzhou_1x1_bc-tyc_18041607_1h	1	3600	These datasets are based on camera data in Hangzhou. Due to the lack of records about turning vehicles, the turning ratios of each dataset are fixed, with 10% as turning left, 60% as going straight, and 30% as turning right. The turning-right vehicles are discarded since they are not under the control of traffic lights. There are one left-turn lane and one straight lane in each direction in each roadnet.	221.03	SOTL
2	hangzhou_1x1_bc-tyc_18041608_1h	1	3600		334.72	SOTL
3	hangzhou_1x1_bc-tyc_18041610_1h	1	3600		213.20	SOTL
4	hangzhou_1x1_kn-hz_18041607_1h	1	3600		72.48	SOTL
5	hangzhou_1x1_kn-hz_18041608_1h	1	3600		64.10	SOTL
6	hangzhou_1x1_qc-yn_18041607_1h	1	3600		117.24	SOTL
7	hangzhou_1x1_qc-yn_18041608_1h	1	3600		131.99	SOTL
8	hangzhou_1x1_sb-sx_18041607_1h	1	3600		173.85	SOTL
9	hangzhou_1x1_sb-sx_18041608_1h	1	3600		290.00	SOTL
10	hangzhou_1x1_tms-xy_18041607_1h	1	3600		214.77	SOTL
11	hangzhou_1x1_tms-xy_18041608_1h	1	3600		325.32	SOTL
12	syn_1x1_uniform_200_1h	1	3600	These datasets are generated artificially. The vehicles enter the road network uniformly with a fixed entering ratio chosen from 200, 400 and 600 vehicles per hour.	61.44	SOTL
13	syn_1x1_uniform_400_1h	1	3600		133.40	SOTL
14	syn_1x1_uniform_600_1h	1	3600		189.11	SOTL
15	hangzhou_4x4_gudang_18010207_1h	16	3600	The road network contains 16 intersections in a 4x4 grid. Each intersection has four incoming approaches and four outgping approaches, and each approach has three lanes (left-turn, through and right-turn respectively). The traffic flow data is based on camera data in Hangzhou. Necessary simplification is done due to the low quality of the real-world data. • Traffic volume: the traffic volume is derived from camera data at Hangzhou. • Turning ratio: 10% (turning left), 60%(going straight) and 30% (turning right). This is synthesized from the statistics of taxi GPS data.	240.97	MaxPressure
16	syn_1x3_gaussian_500_1h	3	3600	The road network contains 16 intersections in a 4x4 grid. Each intersection has four incoming approaches and four outgping approaches, and each approach has three lanes (left-turn, through and right-turn respectively). • Traffic volume: All the vehicles enter and leave the network from the rim edges.For each entering edge, the number of the vehicles generated is sampled from a Gaussian distribution with mean as 500 vehicles/hour/lane. • Turning ratio: 10% (turning left), 60%(going straight) and 30% (turning right)	422.95	MaxPressure
17	syn_2x2_gaussian_500_1h	4	3600		477.71	MaxPressure
18	syn_3x3_gaussian_500_1h	9	3600		631.75	MaxPressure
19	syn_4x4_gaussian_500_1h	16	3600		689.68	MaxPressure
20	Manhattan	2510	3600

Survey

A Survey on traffic signal control

Team

• Zhenhui Jessie Li (Associate professor, Penn State)
• Hua Wei (PhD, Penn State University)
• Guanjie Zheng (PhD, Penn State University)
• Chacha Chen (PhD, Penn State University)
• Nan Xu (PhD, University of Southern California)
• Yuanhao Xiong (PhD, University of Los Angelos)
• Kan Wu (PhD, Penn State University)
• Xinshi Zang (Bachelor, Shanghai Jiao Tong University)
• Huichu Zhang (PhD, Shanghai Jiao Tong University)
• Jie Feng (PhD, Tsinghua University)