This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Deep Double Q-Learning Method for CAVs Traffic Signal Control
Technical Paper
2020-01-5145
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
Urban intersection is the key element to determine the traffic operation of road network. Under the CAVs environment, the roadside control equipment of intersection can communicate with CAVs in real time, collect vehicle state data and optimize traffic control schemes. This paper presents a method for intersection traffic signal control based on deep learning of CAVs data. In addition, intelligent control agent of traffic signal (ICATS) is designed to simulate CAVs. ICATS can perceive real-time changes of traffic flow, model different conditions of intersection and generate the corresponding traffic signal scheme. ICATS used double Q-learning method combination with deep neural network, which is an effective model-independent deep learning algorithm. Moreover, the real traffic data is collected and tested in this paper for evaluating the experiment performance, including vehicle delay, number of passing vehicles, total stop times and passing time. ICATS is compared with another three popular traffic signal control algorithms, like static, actuated and delay_based control algorithm. By means of experimental analysis of real high-flow and low-flow data, the results indicate that signal timing efficiency can be significantly improved by using ICATS method. The experimental results demonstrate that the vehicle delay is shorter, and number of passing vehicles is larger compared with other methods. In this model, the optimal behavior strategy is found depending on trial and error, and the associated state-behavior value is updated by feedback. Finally, by means of this research, ICATS method can more accurately capture the dynamic correlation characteristics of traffic flow and signal timing, and obviously improve the operation efficiency of the intersection.
Authors
Citation
Su, Z., Liu, Q., Zhao, C., and Lin, P., "Deep Double Q-Learning Method for CAVs Traffic Signal Control," SAE Technical Paper 2020-01-5145, 2020, https://doi.org/10.4271/2020-01-5145.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 |
Also In
References
- Ding , J.S.Y. , Li , L. , Peng , H. , and Zhang , Y. Rule-Based Cooperative Merging Strategy for Connected and Automated Vehicles IEEE Transactions on Intelligent Transportation Systems 21 8 3436 3446 2020 https://doi.org/10.1109/tits.2019.2928969
- Rios-Torres , J. and Malikopoulos , A.A. Impact of Connected and Automated Vehicles on Traffic Flow 2017 IEEE 20th International Conference on Intelligent Transportation Systems Japan 2017
- Khayatian , M. , Mehrabian , M. , and Shrivastava , A. RIM: Robust Intersection Management for Connected Autonomous Vehicles 2018 39th IEEE Real-Time Systems Symposium Tunisia 2018
- Zeng , J.H. , Hu , J.M. , and Zhang , Y. Training Reinforcement Learning Agent for Traffic Signal Control under Different Traffic Conditions 2019 IEEE Intelligent Transportation Systems Conference New Zealand 2019
- Wang , Z.R. , Wu , G.Y. , and Barth , M.J. Cooperative Eco-Driving at Signalized Intersections in a Partially Connected and Automated Vehicle Environment IEEE Transactions on Intelligent Transportation Systems 21 5 2029 2038 2020 https://doi.org/10.1109/tits.2019.2911607
- Wang , Y.P. , Cai , P.L. , and Lu , G.Q. Cooperative Autonomous Traffic Organization Method for Connected Automated Vehicles in Multi-Intersection Road Networks Transportation Research Part C-Emerging Technologies 111 458 476 2020 https://doi.org/10.1016/j.trc.2019.12.018
- Wang , Y.L. , Bian , N. , Zhang , L. , and Chen , H. Coordinated Lateral and Longitudinal Vehicle-Following Control of Connected and Automated Vehicles Considering Nonlinear Dynamics IEEE Control Systems Letters 4 4 1054 1059 2020 https://doi.org/10.1109/lcsys.2020.2997994
- Priemer , C. and Friedrich , B. A Decentralized Adaptive Traffic Signal Control Using V2I Communication Data International IEEE Conference on Intelligent Transportation Systems USA 2009
- Feng , Y.L. , Sun , W.L. , and Zheng , J.F. New Dimensions of Intersection Control with Connected and Automated Vehicles the 2nd ACM International Workshop on Smart, Autonomous, and Connected Vehicular Systems and Services USA 2017
- Abdulhai , B. , Pringle , R. , and Karakoulas , G.J. Reinforcement Learning for True Adaptive Traffic Signal Control Journal of Transportation Engineering 129 3 278 285 2003 https://doi.org/10.1061/(asce)0733-947x(2003)129:3(278)
- Jiang , H.F. , Jia , H. , Shi , A. , Meng , W. et al. Eco Approaching at an Isolated Signalized Intersection under Partially Connected and Automated Vehicles Environment Transportation Research Part C-Emerging Technologies 79 290 307 2017 https://doi.org/10.1016/j.trc.2017.04.001
- Arel , I. , Liu , C. , Urbanik , T. , and Kohls , A.G. Reinforcement Learning-Based Multi-Agent System for Network Traffic Signal Control IET Intelligent Transport Systems 4 2 128 135 2010 https://doi.org/10.1049/iet-its.2009.0070
- Zeng , Y.Y. , Xiang , K. , Li , D.S. , and Vasilakos , A.V. Directional Routing and Scheduling for Green Vehicular Delay Tolerant Networks Wireless Networks 19 2 161 173 2013 https://doi.org/10.1007/s11276-012-0457-9
- Shen , X. , Zhang , X.G. , Ouyang , T.H. , Li , Y.C. et al. Cooperative Comfortable-Driving at Signalized Intersections for Connected and Automated Vehicles IEEE Robotics and Automation Letters 5 4 6247 6254 2020 https://doi.org/10.1109/lra.2020.3014010
- Pourmehrab , M. , Emami , P. , Martin-Gasulla , M. , Wilson , J. et al. Signalized Intersection Performance with Automated and Conventional Vehicles: A Comparative Study Journal of Transportation Engineering Part A-Systems 146 9 2020 https://doi.org/10.1061/jtepbs.0000409
- Kashihara , K. Deep Q Learning for Traffic Simulation in Autonomous Driving at a Highway Junction 2017 IEEE International Conference on Systems Canada 2017
- Ge , H.W. , Song , Y.M. , Wu , C.G. , Ren , J.K. et al. Cooperative Deep Q-Learning with Q-Value Transfer for Multi-Intersection Signal Control IEEE Access 7 40797 40809 2019 https://doi.org/10.1109/access.2019.2907618
- Coskun , M. , Abdelkader , B. , and Sanjay , C. Deep Reinforcement Learning for Traffic Light Optimization 2018 18th IEEE International Conference on Data Mining Workshops Singapore 2018
- Zhang , K.P. , Zheng , L. , Liu , Z.J. , and Jia , N. A Deep Learning Based Multitask Model for Network-Wide Traffic Speed Prediction Neurocomputing 396 438 450 2020 https://doi.org/10.1016/j.neucom.2018.10.097