This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Studies on Drivers’ Driving Styles Based on Inverse Reinforcement Learning
Technical Paper
2018-01-0612
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
Although advanced driver assistance systems (ADAS) have been widely introduced in automotive industry to enhance driving safety and comfort, and to reduce drivers’ driving burden, they do not in general reflect different drivers’ driving styles or customized with individual personalities. This can be important to comfort and enjoyable driving experience, and to improved market acceptance. However, it is challenging to understand and further identify drivers’ driving styles due to large number and great variations of driving population. Previous research has mainly adopted physical approaches in modeling drivers’ driving behavior, which however are often very much limited, if not impossible, in capturing human drivers’ driving characteristics. This paper proposes a reinforcement learning based approach, in which the driving styles are formulated through drivers’ learning processes from interaction with surrounding environment. Based on the reinforcement learning theory, driving action can be treated as maximizing a reward function. Instead of calibrating the unknown reward function to satisfy driver’s desired response, we try to recover it from the human driving data, utilizing maximum likelihood inverse reinforcement learning (MLIRL). An IRL-based longitudinal driving assistance system is also proposed in this paper. Firstly, large amount of real world driving data is collected from a test vehicle, and the data is split into two sets for training and for testing purposes respectively. Then, the longitudinal acceleration is modeled as a Boltzmann distribution in human driving activity. The reward function is denoted as a linear combination of some kernelized basis functions. The driving style parameter vector is estimated using MLIRL based on the training set. Finally, a learning-based longitudinal driving assistance algorithm is developed and evaluated on the testing set. The results demonstrate that the proposed method can satisfactorily reflect human drivers’ driving behavior.
Recommended Content
Authors
Topic
Citation
Jiang, Y., Deng, W., Wang, J., and Zhu, B., "Studies on Drivers’ Driving Styles Based on Inverse Reinforcement Learning," SAE Technical Paper 2018-01-0612, 2018, https://doi.org/10.4271/2018-01-0612.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 |
Also In
References
- Bifulco , G.N. , Pariota , L. , Simonelli , F. et al. Development and Testing of a Fully Adaptive Cruise Control System Transportation Research Part C: Emerging Technologies 29 156 170 2013
- Han , D. and Yi , K. A Driver-Adaptive Range Policy for Adaptive Cruise Control Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering 220 3 321 334 2006
- Wang , J. , Zhang , L. , Zhang , D. et al. An Adaptive Longitudinal Driving Assistance System Based on Driver Characteristics IEEE Transactions on Intelligent Transportation Systems 14 1 1 12 2013
- Pariota , L. , Bifulco , G.N. , Galante , F. et al. Longitudinal Control Behaviour: Analysis and Modelling Based on Experimental Surveys in Italy and the UK Accident Analysis & Prevention 89 74 87 2016
- Kesting , A. , Treiber , M. , Schönhof , M. et al. Adaptive Cruise Control Design for Active Congestion Avoidance Transportation Research Part C: Emerging Technologies 16 6 668 683 2008
- Moon , S. , Moon , I. , and Yi , K. Design, Tuning, and Evaluation of a Full-Range Adaptive Cruise Control System with Collision Avoidance Control Engineering Practice 17 4 442 455 2009
- Moon , S. and Yi , K. Human Driving Data-Based Design of a Vehicle Adaptive Cruise Control Algorithm Vehicle System Dynamics 46 8 661 690 2008
- Okuda , H. , Ikami , N. , Suzuki , T. et al. Modeling and Analysis of Driving Behavior Based on a Probability-Weighted ARX Model IEEE Transactions on Intelligent Transportation Systems 14 1 98 112 2013
- Okuda , H. , Guo , X. , Tazaki , Y. et al. Model predictive driver assistance control for cooperative cruise based on hybrid system driver model[C] American Control Conference (ACC) 2014 4630 4636
- Gote C , Flad M , Hohmann S Driver characterization & driver specific trajectory planning: an inverse optimal control approach[C] Systems, Man and Cybernetics (SMC) 2014 IEEE International Conference on 3014 3021
- Butakov , V.A. and Ioannou , P. Personalized Driver Assistance for Signalized Intersections Using V2I Communication IEEE Transactions on Intelligent Transportation Systems 17 7 1910 1919 2016
- Xu , L. , Hu , J. , Jiang , H. et al. Establishing Style-Oriented Driver Models by Imitating Human Driving Behaviors IEEE Transactions on Intelligent Transportation Systems 16 5 2522 2530 2015
- Butakov , V.A. and Ioannou , P. Personalized Driver/Vehicle Lane Change Models for ADAS IEEE Transactions on Vehicular Technology 64 10 4422 4431 2015
- Angkititrakul , P. , Miyajima , C. , and Takeda K. Modeling and Adaptation of Stochastic Driver-Behavior Model with Application to Car Following Intelligent Vehicles Symposium (IV), 2011 IEEE 2011 814 819
- Zhao , D. , Hu , Z. , Xia , Z. et al. Full-Range Adaptive Cruise Control Based on Supervised Adaptive Dynamic Programming Neurocomputing 125 57 67 2014
- Lefèvre , S. , Carvalho , A. , and Borrelli , F. A Learning-Based Framework for Velocity Control in Autonomous Driving IEEE Transactions on Automation Science and Engineering 13 1 32 42 2016
- Gindele , T. , Brechtel , S. , and Dillmann , R. Learning Driver Behavior Models from Traffic Observations for Decision Making and Planning IEEE Intelligent Transportation Systems Magazine 7 1 69 79 2015
- Gadepally , V.N. 2013
- Ng , A.Y. and Russell , S.J. Algorithms for Inverse Reinforcement Learning ICML 2000 663 670
- Abbeel , P. and Ng , A.Y. Apprenticeship Learning via Inverse Reinforcement Learning Proceedings of the Twenty-First International Conference on Machine Learning 2004
- Ziebart , B.D. , Maas , A.L. , Bagnell , J.A. et al. 2008 1433 1438
- Wilson , R.E. Mechanisms for Spatio-Temporal Pattern Formation in Highway Traffic Models Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 366 1872 2017 2032 2008
- Ormoneit , D. and Sen , Ś. Kernel-Based Reinforcement Learning Machine Learning 49 2 161 178 2002