Open Access

Robust Multiagent Reinforcement Learning toward Coordinated Decision-Making of Automated Vehicles

Journal Article
10-07-04-0031
ISSN: 2380-2162, e-ISSN: 2380-2170
Published September 04, 2023 by SAE International in United States
Robust Multiagent Reinforcement Learning toward Coordinated
                    Decision-Making of Automated Vehicles
Sector:
Citation: He, X., Chen, H., and Lv, C., "Robust Multiagent Reinforcement Learning toward Coordinated Decision-Making of Automated Vehicles," SAE Int. J. Veh. Dyn., Stab., and NVH 7(4):475-488, 2023, https://doi.org/10.4271/10-07-04-0031.
Language: English

References

  1. Lai , F. , Huang , C. , Jiang , C. , and Zhang , Y. Simulation Analysis of Automatic Emergency Braking System under Constant Steer Conditions SAE Int. J. Veh. Dyn., Stab., and NVH 6 4 461 476 2022 https://doi.org/10.4271/10-06-04-0030
  2. Ren , Y. , Jiang , J. , Zhan , G. , Li , S.E. et al. Self-Learned Intelligence for Integrated Decision and Control of Automated Vehicles at Signalized Intersections IEEE Transactions on Intelligent Transportation Systems 23 12 24145 24156 2022
  3. Wang , Y. , Wei , H. , Hu , B. , and Lv , C. Robust Estimation of Vehicle Dynamic State Using a Novel Second-Order Fault-Tolerant Extended Kalman Filter SAE Int. J. Veh. Dyn., Stab., and NVH 7 3 2023 https://doi.org/10.4271/10-07-03-0019
  4. Mnih , V. , Kavukcuoglu , K. , Silver , D. , Rusu , A.A. et al. Human-Level Control through Deep Reinforcement Learning Nature 518 7540 529 533 2015
  5. Vaswani , A. , Shazeer , N. , Parmar , N. , Uszkoreit , J. et al. Attention Is All You Need Advances in Neural Information Processing Systems 30 1 11 2017
  6. Zhao , Q. , Zheng , H. , Kaku , C. , Cheng , F. , and Zong , C. Safety Spacing Control of Truck Platoon Based on Emergency Braking under Different Road Conditions SAE Int. J. Veh. Dyn., Stab., and NVH 7 1 69 81 2023 https://doi.org/10.4271/10-07-01-0005
  7. Cao , Z. , Xu , S. , Peng , H. , Yang , D. , and Zidek , R. Confidence-Aware Reinforcement Learning for Self-Driving Cars IEEE Transactions on Intelligent Transportation Systems 23 7 7419 7430 2022
  8. Gupta , U. , Nouri , A. , Subramanian , C. , Taheri , S. et al. Developing an Experimental Setup for Real-Time Road Surface Identification Using Intelligent Tires SAE Int. J. Veh. Dyn., Stab., and NVH 5 3 351 367 2021 https://doi.org/10.4271/10-05-03-0024
  9. Ji , X. , He , X. , Lv , C. , Liu , Y. , and Wu , J. Adaptive-Neural-Network-Based Robust Lateral Motion Control for Autonomous Vehicle at Driving Limits Control Engineering Practice 76 41 53 2018
  10. Peng , H. and Chen , X. Active Safety Control of X-by-Wire Electric Vehicles: A Survey SAE Int. J. Veh. Dyn., Stab., and NVH 6 2 115 133 2022 https://doi.org/10.4271/10-06-02-0008
  11. Peng , J. , Zhang , S. , Zhou , Y. , and Li , Z. An Integrated Model for Autonomous Speed and Lane Change Decision-Making Based on Deep Reinforcement Learning IEEE Transactions on Intelligent Transportation Systems 23 11 21848 21860 2022
  12. Wang , Y. , Wei , H. , Hu , B. , and Lv , C. A Review of Dynamic State Estimation of the Neighborhood System for Connected Vehicles SAE Int. J. Veh. Dyn., Stab., and NVH 7 3 2023 https://doi.org/10.4271/10-07-03-0023
  13. Negash , N.M. and Yang , J. Anticipation-Based Autonomous Platoon Control Strategy with Minimum Parameter Learning Adaptive Radial Basis Function Neural Network Sliding Mode Control SAE Int. J. Veh. Dyn., Stab., and NVH 6 3 247 265 2022 https://doi.org/10.4271/10-06-03-0017
  14. Wu , J. , Zhang , J. , Nie , B. , Liu , Y. , and He , X. Adaptive Control of PMSM Servo System for Steering-by-Wire System with Disturbances Observation IEEE Transactions on Transportation Electrification 8 2 2015 2028 2021
  15. Schwarting , W. , Alonso-Mora , J. , and Rus , D. Planning and Decision-Making for Autonomous Vehicles Annual Review of Control, Robotics, and Autonomous Systems 1 187 210 2018
  16. Urmson , C. , Anhalt , J. , Bagnell , D. , Baker , C. et al. Autonomous Driving in Urban Environments: Boss and the Urban Challenge Journal of Field Robotics 25 8 425 466 2008
  17. Montemerlo , M. , Becker , J. , Bhat , S. , Dahlkamp , H. et al. Junior: The Stanford Entry in the Urban Challenge Journal of Field Robotics 25 9 569 597 2008
  18. Kurt , A. and Özgüner , Ü. Hierarchical Finite State Machines for Autonomous Mobile Systems Control Engineering Practice 21 2 184 194 2013
  19. Sales , D.O. , Correa , D.O. , Fernandes , L.C. , Wolf , D.F. , and Osório , F.S. Adaptive Finite State Machine Based Visual Autonomous Navigation System Engineering Applications of Artificial Intelligence 29 152 162 2014
  20. Hülnhagen , T. , Dengler , I. , Tamke , A. , Dang , T. et al. Maneuver Recognition Using Probabilistic Finite-State Machines and Fuzzy Logic 2010 IEEE Intelligent Vehicles Symposium La Jolla, CA 65 70 2010
  21. Li , N. , Oyler , D.W. , Zhang , M. , Yildiz , Y. et al. Game Theoretic Modeling of Driver and Vehicle Interactions for Verification and Validation of Autonomous Vehicle Control Systems IEEE Transactions on Control Systems Technology 26 5 1782 1797 2017
  22. Hang , P. , Lv , C. , Xing , Y. , Huang , C. , and Hu , Z. Human-Like Decision Making for Autonomous Driving: A Noncooperative Game Theoretic Approach IEEE Transactions on Intelligent Transportation Systems 22 4 2076 2087 2020
  23. Liu , W. , Kim , S.-W. , Pendleton , S. , and Ang , M. H. Situation-Aware Decision Making for Autonomous Driving on Urban Road Using Online POMDP 2015 IEEE Intelligent Vehicles Symposium (IV) Seoul, Korea 1126 1133 2015
  24. Bai , H. , Cai , S. , Ye , N. , Hsu , D. et al. Intention-Aware Online POMDP Planning for Autonomous Driving in a Crowd 2015 IEEE International Conference on Robotics and Automation (ICRA) Seattle, WA 454 460 2015
  25. Le Mero , L. , Yi , D. , Dianati , M. , and Mouzakitis , A. A Survey on Imitation Learning Techniques for End-to-End Autonomous Vehicles IEEE Transactions on Intelligent Transportation Systems 23 9 14128 14147 2022
  26. Chen , L. , He , Y. , Wang , Q. , Pan , W. , and Ming , Z. Joint Optimization of Sensing, Decision-Making and Motion-Controlling for Autonomous Vehicles: A Deep Reinforcement Learning Approach IEEE Transactions on Vehicular Technology 71 5 4642 4654 2022
  27. Kuefler , A. , Morton , J. , Wheeler , T. , and Kochenderfer , M. Imitating Driver Behavior with Generative Adversarial Networks 2017 IEEE Intelligent Vehicles Symposium (IV) Los Angeles, CA 204 211 2017
  28. Ngai , D.C.K. and Yung , N.H.C. A Multiple-Goal Reinforcement Learning Method for Complex Vehicle Overtaking Maneuvers IEEE Transactions on Intelligent Transportation Systems 12 2 509 522 2011
  29. Chen , J. , Wang , Z. , and Tomizuka , M. Deep Hierarchical Reinforcement Learning for Autonomous Driving with Distinct Behaviors 2018 IEEE Intelligent Vehicles Symposium (IV) Changshu, China 1239 1244 2018
  30. Everett , M. , Chen , Y. F. , and How , J. P. Motion Planning among Dynamic, Decision-Making Agents with Deep Reinforcement Learning 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Madrid, Spain 3052 3059 2018
  31. Xu , X. , Zuo , L. , Li , X. , Qian , L. et al. A Reinforcement Learning Approach to Autonomous Decision Making of Intelligent Vehicles on Highways IEEE Transactions on Systems, Man, and Cybernetics: Systems 50 10 3884 3897 2018
  32. You , C. , Lu , J. , Filev , D. , and Tsiotras , P. Advanced Planning for Autonomous Vehicles Using Reinforcement Learning and Deep Inverse Reinforcement Learning Robotics and Autonomous Systems 114 1 18 2019
  33. Mozaffari , S. , Arnold , E. , Dianati , M. , and Fallah , S. Early Lane Change Prediction for Automated Driving Systems Using Multi-Task Attention-Based Convolutional Neural Networks IEEE Transactions on Intelligent Vehicles 7 3 758 770 2022
  34. Szegedy , C. , Toshev , A. , and Erhan , D. Deep Neural Networks for Object Detection Advances in Neural Information Processing Systems 26 1 9 2013
  35. Hu , Z. , Xing , Y. , Gu , W. , Cao , D. , and Lv , C. Driver Anomaly Quantification for Intelligent Vehicles: A Contrastive Learning Approach with Representation Clustering IEEE Transactions on Intelligent Vehicles 8 1 37 47 2023
  36. Xiao , Y. , Codevilla , F. , Gurram , A. , Urfalioglu , O. , and López , A.M. Multimodal End-to-End Autonomous Driving IEEE Transactions on Intelligent Transportation Systems 23 1 537 547 2022
  37. He , X. , Yang , H. , Hu , Z. , and Lv , C. Robust Lane Change Decision Making for Autonomous Vehicles: An Observation Adversarial Reinforcement Learning Approach IEEE Transactions on Intelligent Vehicles 8 1 184 193 2023
  38. Zhang , J. , Chang , C. , Zeng , X. , and Li , L. Multi-Agent DRL-Based Lane Change with Right-of-Way Collaboration Awareness IEEE Transactions on Intelligent Transportation Systems 24 1 854 869 2023
  39. Lopez , P. A. , Behrisch , M. , Bieker-Walz , L. , Erdmann , J. et al. Microscopic Traffic Simulation Using Sumo The 21st IEEE International Conference on Intelligent Transportation Systems Maui, HI 2018
  40. Boyd , S. , Boyd , S.P. , and Vandenberghe , L. Convex Optimization Cambridge Cambridge University Press 2004
  41. Crosato , L. , Shum , H.P.H. , Ho , E.S.L. , and Wei , C. Interaction-Aware Decision-Making for Automated Vehicles Using Social Value Orientation IEEE Transactions on Intelligent Vehicles 8 2 1339 1349 2022
  42. He , X. , Liu , Y. , Lv , C. , Ji , X. , and Liu , Y. Emergency Steering Control of Autonomous Vehicle for Collision Avoidance and Stabilisation Vehicle System Dynamics 57 8 1163 1187 2019
  43. He , X. , Lou , B. , Yang , H. , and Lv , C. Robust Decision Making for Autonomous Vehicles at Highway On-Ramps: A Constrained Adversarial Reinforcement Learning Approach IEEE Transactions on Intelligent Transportation Systems 24 4 4103 4113 2022
  44. Rajamani , R. Vehicle Dynamics and Control New York Springer Science & Business Media 2011
  45. Yu , C. , Velu , A. , Vinitsky , E. , Gao , J. et al. The Surprising Effectiveness of PPO in Cooperative Multi-Agent Games Thirty-Sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track New Orleans, Louisiana, USA 2022
  46. Haarnoja , T. , Zhou , A. , Abbeel , P. , and Levine , S. Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor International Conference on Machine Learning Stockholm, Sweden 1861 1870 2018
  47. Yuan , W. , Zhuang , H. , Wang , C. , and Yang , M. AGBM: An Adaptive Gradient Balanced Mechanism for the End-to-End Steering Estimation IEEE Transactions on Intelligent Transportation Systems 23 9 16016 16025 2022

Cited By