This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Lane Change Decision Algorithm Based on Deep Q Network for Autonomous Vehicles
Technical Paper
2022-01-0084
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
For high levels autonomous driving functions, the Decision Layer often takes on more responsibility due to the requirement of facing more diverse and even rare conditions. It is very difficult to accurately find a safe and efficient lane change timing when autonomous vehicles encounter complex traffic flow and need to change lanes. The traditional method based on rules and experiences has the limitation that it is difficult to be taken into account all possible conditions. Therefore, this paper designs a lane-changing decision algorithm based on data-driven and machine learning, and uses the DQN (Deep Q Network) algorithm in Reinforcement Learning to determine the appropriate lane-changing timing and target lane. Firstly, the scene characteristics of the highway are analyzed, the input and output of the decision-making model are designated and the data from the Perception Layer are processed. The DQN network architecture is built, and the reward function is designated considering multi-objective requirements. Then, a virtual scene is built in PreScan, and random factors are added to simulate various scenarios that vehicles might encounter when making lane change decisions. Based on this scenario, the Reinforcement Learning model is trained. Finally, a bench test comparing between the trained Reinforcement Learning model and rule-based model and a real vehicle test on a highway are carried out to verify the effectiveness of the algorithm. By analyzing the results of both bench test and real vehicle test, it can be concluded that the lane-changing decision model based on Reinforcement Learning not only improves driving efficiency but also ensures safety under subjective and objective assessment, and has better adaptability to different working conditions, which is closer to human decision.
Authors
Citation
Zhang, X., Liu, X., Li, X., and Wu, G., "Lane Change Decision Algorithm Based on Deep Q Network for Autonomous Vehicles," SAE Technical Paper 2022-01-0084, 2022, https://doi.org/10.4271/2022-01-0084.Also In
References
- Xueyun , L. , Shuang , L. , and Zhang , J. Robust Controller Design for Trajectory Tracking of Autonomous Vehicle International Journal of Vehicle Performance 6 4 2020 2020 10.1504/IJVP.2020.111406
- Pandey , V. , Slon , C. , Deschenes , L. , Gorsich , D. et al. A Decision Based Mobility Model for Semi and Fully Autonomous Vehicles SAE Int. J. Adv. & Curr. Prac. in Mobility 2 5 2020 2640 2649 10.1109/TITS.2019.2913998
- Montemerlo , M. , Becker , J. , Bhat , S. et al. Junior: The Stanford Entry in the Urban Challenge Journal of Field ROBOTICS 25 9 2008 569 597 10.4271/2020-01-0747
- Gindele , T. , Jagszent , D. , Pitzer , B. , et al Design of the Planner of Team AnnieWAY’s Autonomous Vehicle Used in the DARPA Urban Challenge 2007 Intelligent Vehicles Symposium, 2008 IEEE IEEE 2008 1131 1136 10.1109/IVS.2008.4621268
- Baker , C.R. and Dolan , J.M. Traffic Interaction in the Urban Challenge: Putting Boss on Its Best Behavior IROS 2008 Intelligent Robots and Systems IEEE/RSJ International Conference on IEEE 2008 2008 1752 1758 10.1109/IROS.2008.4651211
- Bojarski , M. , Del Testa , D. , Dworakowski , D. , et al. 2016
- Bansal , M. , Krizhevsky , A. , Ogale , A. 2018
- Ngai , D.C.K. , Yung et al. A Multiple-Goal Reinforcement Learning Method for Complex Vehicle Overtaking Maneuvers Intelligent Transportation Systems IEEE Transactions on 2011 10.1109/TITS.2011.2106158
- Wang , P. , Chan , C.Y. , Fortelle , A. A Reinforcement Learning Based Approach for Automated Lane Change Maneuvers 2018 IEEE Intelligent Vehicles Symposium (IV) IEEE 2018 10.1109/IVS.2018.8500556
- Ren , M. and Guangqiang , W. Integrated Strategy For Automatic Lane-Changing Decision and Trajectory Planning in Dynamic Scenario IDETC/CIE 2020 2020 10.1115/DETC2020-22294
- Hoel , C.J. , Wolff , K. , Laine , L. Automated Speed and Lane Change Decision Making Using Deep Reinforcement Learning 2018 21st International Conference on Intelligent Transportation Systems (ITSC), IEEE 2018 10.1109/ITSC.2018.8569568
- Sutton , R.S. and Barto , A.G. Introduction to Reinforcement Learning MIT Press 1998
- Mnih , V. , Kavukcuoglu , K. , Silver , D. et al. Playing Atari with Deep Reinforcement Learning Computer Science 2013
- Volodymyr , M. , Koray , K. , David , S. et al. Human-Level Control through Deep Reinforcement Learning Nature 518 7540 2019 529 533
- Treiber , M. , Hennecke , A. , and Helbing , D. Congested Traffic States in Empirical Observations and Microscopic Simulations Phys. Rev. E 62 2000 1805 1824 10.1103/PhysRevE.62.1805
- Kesting , A. , Treiber , M. , and Helbing , D. General Lane-Changing Model Mobil for Car-Following Models Transportation Research Record 1999 2007 86 94 10.3141/1999-10