This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Studies on Drivers’ Driving Styles Based on Inverse Reinforcement Learning
ISSN: 0148-7191, e-ISSN: 2688-3627
Published April 3, 2018 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
Although advanced driver assistance systems (ADAS) have been widely introduced in automotive industry to enhance driving safety and comfort, and to reduce drivers’ driving burden, they do not in general reflect different drivers’ driving styles or customized with individual personalities. This can be important to comfort and enjoyable driving experience, and to improved market acceptance. However, it is challenging to understand and further identify drivers’ driving styles due to large number and great variations of driving population. Previous research has mainly adopted physical approaches in modeling drivers’ driving behavior, which however are often very much limited, if not impossible, in capturing human drivers’ driving characteristics. This paper proposes a reinforcement learning based approach, in which the driving styles are formulated through drivers’ learning processes from interaction with surrounding environment. Based on the reinforcement learning theory, driving action can be treated as maximizing a reward function. Instead of calibrating the unknown reward function to satisfy driver’s desired response, we try to recover it from the human driving data, utilizing maximum likelihood inverse reinforcement learning (MLIRL). An IRL-based longitudinal driving assistance system is also proposed in this paper. Firstly, large amount of real world driving data is collected from a test vehicle, and the data is split into two sets for training and for testing purposes respectively. Then, the longitudinal acceleration is modeled as a Boltzmann distribution in human driving activity. The reward function is denoted as a linear combination of some kernelized basis functions. The driving style parameter vector is estimated using MLIRL based on the training set. Finally, a learning-based longitudinal driving assistance algorithm is developed and evaluated on the testing set. The results demonstrate that the proposed method can satisfactorily reflect human drivers’ driving behavior.
CitationJiang, Y., Deng, W., Wang, J., and Zhu, B., "Studies on Drivers’ Driving Styles Based on Inverse Reinforcement Learning," SAE Technical Paper 2018-01-0612, 2018, https://doi.org/10.4271/2018-01-0612.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
- Bifulco, G.N., Pariota, L., Simonelli, F. et al., “Development and Testing of a Fully Adaptive Cruise Control System,” Transportation Research Part C: Emerging Technologies 29:156-170, 2013.
- Han, D. and Yi, K., “A Driver-Adaptive Range Policy for Adaptive Cruise Control,” Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering 220(3):321-334, 2006.
- Wang, J., Zhang, L., Zhang, D. et al., “An Adaptive Longitudinal Driving Assistance System Based on Driver Characteristics,” IEEE Transactions on Intelligent Transportation Systems 14(1):1-12, 2013.
- Pariota, L., Bifulco, G.N., Galante, F. et al., “Longitudinal Control Behaviour: Analysis and Modelling Based on Experimental Surveys in Italy and the UK,” Accident Analysis & Prevention 89:74-87, 2016.
- Kesting, A., Treiber, M., Schönhof, M. et al., “Adaptive Cruise Control Design for Active Congestion Avoidance,” Transportation Research Part C: Emerging Technologies 16(6):668-683, 2008.
- Moon, S., Moon, I., and Yi, K., “Design, Tuning, and Evaluation of a Full-Range Adaptive Cruise Control System with Collision Avoidance,” Control Engineering Practice 17(4):442-455, 2009.
- Moon, S. and Yi, K., “Human Driving Data-Based Design of a Vehicle Adaptive Cruise Control Algorithm,” Vehicle System Dynamics 46(8):661-690, 2008.
- Okuda, H., Ikami, N., Suzuki, T. et al., “Modeling and Analysis of Driving Behavior Based on a Probability-Weighted ARX Model,” IEEE Transactions on Intelligent Transportation Systems 14(1):98-112, 2013.
- Okuda, H., Guo, X., Tazaki, Y. et al., “Model predictive driver assistance control for cooperative cruise based on hybrid system driver model[C]American Control Conference (ACC), 2014. IEEE, 2014: 4630-4636.
- Gote C, Flad M, Hohmann S. Driver characterization & driver specific trajectory planning: an inverse optimal control approach[C] Systems, Man and Cybernetics (SMC) , 2014IEEE International Conference on IEEE, 2014: 3014-3021.
- Butakov, V.A. and Ioannou, P., “Personalized Driver Assistance for Signalized Intersections Using V2I Communication,” IEEE Transactions on Intelligent Transportation Systems 17(7):1910-1919, 2016.
- Xu, L., Hu, J., Jiang, H. et al., “Establishing Style-Oriented Driver Models by Imitating Human Driving Behaviors,” IEEE Transactions on Intelligent Transportation Systems 16(5):2522-2530, 2015.
- Butakov, V.A. and Ioannou, P., “Personalized Driver/Vehicle Lane Change Models for ADAS,” IEEE Transactions on Vehicular Technology 64(10):4422-4431, 2015.
- Angkititrakul, P., Miyajima, C., and Takeda K., “Modeling and Adaptation of Stochastic Driver-Behavior Model with Application to Car Following,” Intelligent Vehicles Symposium (IV), 2011 IEEE, IEEE, 2011, 814-819.
- Zhao, D., Hu, Z., Xia, Z. et al., “Full-Range Adaptive Cruise Control Based on Supervised Adaptive Dynamic Programming,” Neurocomputing 125:57-67, 2014.
- Lefèvre, S., Carvalho, A., and Borrelli, F., “A Learning-Based Framework for Velocity Control in Autonomous Driving,” IEEE Transactions on Automation Science and Engineering 13(1):32-42, 2016.
- Gindele, T., Brechtel, S., and Dillmann, R., “Learning Driver Behavior Models from Traffic Observations for Decision Making and Planning,” IEEE Intelligent Transportation Systems Magazine 7(1):69-79, 2015.
- Gadepally, V.N., “Estimation of Driver Behavior for Autonomous Vehicle Applications,” The Ohio State University, 2013.
- Ng, A.Y. and Russell, S.J., “Algorithms for Inverse Reinforcement Learning,” ICML, 2000, 663-670, www.
- Abbeel, P. and Ng, A.Y., “Apprenticeship Learning via Inverse Reinforcement Learning,” Proceedings of the Twenty-First International Conference on Machine Learning, ACM, 2004.
- Ziebart, B.D., Maas, A.L., Bagnell, J.A. et al., “Maximum Entropy Inverse Reinforcement Learning,” AAAI, 2008, 8, 1433-1438.
- Wilson, R.E., “Mechanisms for Spatio-Temporal Pattern Formation in Highway Traffic Models,” Philosophical Transactions of the Royal Society of London A: Mathematical, Physical and Engineering Sciences 366(1872):2017-2032, 2008.
- Ormoneit, D. and Sen, Ś., “Kernel-Based Reinforcement Learning,” Machine Learning 49(2):161-178, 2002.