This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Real-Time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle
Technical Paper
2019-01-1208
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
Energy management of hybrid vehicle has been a widely researched area. Strategies like dynamic programming (DP), equivalent consumption minimization strategy (ECMS), Pontryagin’s minimum principle (PMP) are well analyzed in literatures. However, the adaptive optimization work is still lacking, especially for reinforcement learning (RL). In this paper, Q-learning, as one of the model-free reinforcement learning method, is implemented in a mid-size 48V mild parallel hybrid electric vehicle (HEV) framework to optimize the fuel economy. Different from other RL work in HEV, this paper only considers vehicle speed and vehicle torque demand as the Q-learning states. SOC is not included for the reduction of state dimension. This paper focuses on showing that the EMS with non-SOC state vectors are capable of controlling the vehicle and outputting satisfactory results. Electric motor torque demand is chosen as action. In the reward function definition, the fuel consumption contains engine fuel consumption and equivalent battery fuel consumption. The Q-learning table is executed over Worldwide Harmonized Light Vehicle driving cycle. The learning process shows fast convergence and fuel economy improvement of 5%. The Q-learning strategy is compared with baseline thermostatic rule-based EMS and ECMS. The Q-learning result shows 8.89% fuel economy improvement than the rule-based EMS and 0.88% than the ECMS.
Recommended Content
Authors
Citation
Xu, B., Malmir, F., Rathod, D., and Filipi, Z., "Real-Time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle," SAE Technical Paper 2019-01-1208, 2019, https://doi.org/10.4271/2019-01-1208.Data Sets - Support Documents
| Title | Description | Download |
|---|---|---|
| [Unnamed Dataset 1] | ||
| [Unnamed Dataset 2] | ||
| [Unnamed Dataset 3] |
Also In
References
- Sciarretta, A., Back, M., and Guzzella, L., “Optimal Control of Parallel Hybrid Electric Vehicles,” IEEE Transactions on Control Systems Technology 12:352-363, 2004.
- Bellman, R., Dynamic Programming (Courier Corporation, 2013).
- Onori, S., Serrao, L., and Rizzoni, G., “Pontryagin’s Minimum Principle,” in: Hybrid Electric Vehicles (Springer, 2016), 51-63.
- Ross, S.M., Introduction to Stochastic Dynamic Programming (Academic Press, 2014).
- Sutton, R.S., Barto, A.G., and Bach, F., Reinforcement Learning: An Introduction (MIT Press, 1998).
- Kober, J., Bagnell, J.A., and Peters, J., “Reinforcement Learning in Robotics: A Survey,” The International Journal of Robotics Research 32:1238-1274, 2013.
- Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A. et al., “Human-Level Control through Deep Reinforcement Learning,” Nature 518:529, 2015.
- Theocharous, G., Thomas, P.S., and Ghavamzadeh, M., “Personalized Ad Recommendation Systems for Life-Time Value Optimization with Guarantees,” in: IJCAI, 2015, 1806-1812.
- Xiong, R., Cao, J., and Yu, Q., “Reinforcement Learning-Based Real-Rime Power Management for Hybrid Energy Storage System in the Plug-In Hybrid Electric Vehicle,” Applied Energy 211:538-548, 2018.
- Fang, Y., Song, C., Xia, B., and Song, Q., “An Energy Management Strategy for Hybrid Electric Bus Based on Reinforcement Learning,” in Control and Decision Conference (CCDC), 2015 27th Chinese, 2015, 4973-4977.
- Wu, J., He, H., Peng, J., Li, Y. et al., “Continuous Reinforcement Learning of Energy Management with Deep Q Network for a Power Split Hybrid Electric Bus,” Applied Energy 222:799-811, 2018.
- Argonne, “Autonomie,” 2018, http://www.autonomie.net/.
- Malmir, F., Xu, B., and Filipi, Z., “A Heuristic Supervisory Controller for a 48V Hybrid Electric Vehicle Considering Fuel Economy and Battery Aging,” SAE Technical Paper 2019-01-0079, 2019, doi:10.4271/2019-01-0079.
- Watkins, C.J.C.H., Learning from Delayed Rewards (Cambridge: King’s College, 1989).
- Zou, Y., Liu, T., Liu, D., and Sun, F., “Reinforcement Learning-Based Real-Time Energy Management for a Hybrid Tracked Vehicle,” Applied Energy 171:372-382, 2016.