This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Real-Time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle
Technical Paper
2019-01-1208
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
Energy management of hybrid vehicle has been a widely researched area. Strategies like dynamic programming (DP), equivalent consumption minimization strategy (ECMS), Pontryagin’s minimum principle (PMP) are well analyzed in literatures. However, the adaptive optimization work is still lacking, especially for reinforcement learning (RL). In this paper, Q-learning, as one of the model-free reinforcement learning method, is implemented in a mid-size 48V mild parallel hybrid electric vehicle (HEV) framework to optimize the fuel economy. Different from other RL work in HEV, this paper only considers vehicle speed and vehicle torque demand as the Q-learning states. SOC is not included for the reduction of state dimension. This paper focuses on showing that the EMS with non-SOC state vectors are capable of controlling the vehicle and outputting satisfactory results. Electric motor torque demand is chosen as action. In the reward function definition, the fuel consumption contains engine fuel consumption and equivalent battery fuel consumption. The Q-learning table is executed over Worldwide Harmonized Light Vehicle driving cycle. The learning process shows fast convergence and fuel economy improvement of 5%. The Q-learning strategy is compared with baseline thermostatic rule-based EMS and ECMS. The Q-learning result shows 8.89% fuel economy improvement than the rule-based EMS and 0.88% than the ECMS.
Recommended Content
Authors
Citation
Xu, B., Malmir, F., Rathod, D., and Filipi, Z., "Real-Time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle," SAE Technical Paper 2019-01-1208, 2019, https://doi.org/10.4271/2019-01-1208.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 | ||
Unnamed Dataset 2 | ||
Unnamed Dataset 3 |
Also In
References
- Sciarretta , A. , Back , M. , and Guzzella , L. Optimal Control of Parallel Hybrid Electric Vehicles IEEE Transactions on Control Systems Technology 12 352 363 2004
- Bellman , R. Dynamic Programming Courier Corporation 2013
- Onori , S. , Serrao , L. , and Rizzoni , G. Pontryagin’s Minimum Principle Hybrid Electric Vehicles Springer 2016 51 63
- Ross , S.M. Introduction to Stochastic Dynamic Programming Academic Press 2014
- Sutton , R.S. , Barto , A.G. , and Bach , F. Reinforcement Learning: An Introduction MIT Press 1998
- Kober , J. , Bagnell , J.A. , and Peters , J. Reinforcement Learning in Robotics: A Survey The International Journal of Robotics Research 32 1238 1274 2013
- Mnih , V. , Kavukcuoglu , K. , Silver , D. , Rusu , A.A. et al. Human-Level Control through Deep Reinforcement Learning Nature 518 529 2015
- Theocharous , G. , Thomas , P.S. , and Ghavamzadeh , M. Personalized Ad Recommendation Systems for Life-Time Value Optimization with Guarantees IJCAI 2015 1806 1812
- Xiong , R. , Cao , J. , and Yu , Q. Reinforcement Learning-Based Real-Rime Power Management for Hybrid Energy Storage System in the Plug-In Hybrid Electric Vehicle Applied Energy 211 538 548 2018
- Fang , Y. , Song , C. , Xia , B. , and Song , Q. An Energy Management Strategy for Hybrid Electric Bus Based on Reinforcement Learning Control and Decision Conference (CCDC), 2015 27th Chinese 2015 4973 4977
- Wu , J. , He , H. , Peng , J. , Li , Y. et al. Continuous Reinforcement Learning of Energy Management with Deep Q Network for a Power Split Hybrid Electric Bus Applied Energy 222 799 811 2018
- Argonne Autonomie 2018 http://www.autonomie.net/
- Malmir , F. , Xu , B. , and Filipi , Z. A Heuristic Supervisory Controller for a 48V Hybrid Electric Vehicle Considering Fuel Economy and Battery Aging SAE Technical Paper 2019-01-0079 2019 10.4271/2019-01-0079
- Watkins , C.J.C.H. Learning from Delayed Rewards Cambridge King’s College 1989
- Zou , Y. , Liu , T. , Liu , D. , and Sun , F. Reinforcement Learning-Based Real-Time Energy Management for a Hybrid Tracked Vehicle Applied Energy 171 372 382 2016