This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Reinforcement Learning Based Energy Management of Hybrid Energy Storage Systems in Electric Vehicles
Technical Paper
2021-01-0197
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Event:
SAE WCX Digital Summit
Language:
English
Abstract
Energy management in electric vehicles plays a significant role in both reducing energy consumption and limiting the rate of battery capacity degradation. It is especially important for systems with multiple energy storage units where optimally arbitrating power demand among the energy storage units is challenging. While many optimal control methods exist for designing a good energy management system, in this work a Reinforcement-Learning (RL) methodology is explored to design an energy management system for an electric vehicle with a Hybrid Energy Storage System (HESS) that included a battery and a supercapacitor. The energy management system is designed to optimally divide the traction power request among a battery and a super-capacitor in real-time; while trying to minimize the overall energy consumption and battery degradation. Along with a vehicle and electric propulsion model, an empirical battery aging model was incorporated in the problem formulation to address effect of energy management on battery capacity degradation. Several optimization objectives including total energy consumption and long-term battery health impact were incorporated to the design to devise and further adapt a power arbitration strategy based on current HESS states and the total power demand to be achieved. The design process is presented starting from initial modeling work and various reinforcement learning formulations as well as training and validation of the RL-based energy management system utilizing developed models and with thousands of kilometers of real-world driving profiles in a high-performance computing (HPC) environment. It was demonstrated that one can successfully incorporate various heuristic design objectives (e.g. minimize loss for EV range, reduce battery throughput for aging) into the optimization problem in a data-driven framework. It was shown through a simulation study that battery aging can be reduced by about 4% for the electric vehicle considered in this work.
Recommended Content
Authors
Topic
Citation
Haskara, I., Hegde, B., and Chang, C., "Reinforcement Learning Based Energy Management of Hybrid Energy Storage Systems in Electric Vehicles," SAE Technical Paper 2021-01-0197, 2021, https://doi.org/10.4271/2021-01-0197.Also In
References
- Tang , L. , Rizzoni , G. , and Onori , S. Energy Management Strategy for HEVs Including Battery Life Optimization IEEE Transactions on Transportation Electrification 211 222 Oct. 2015 10.1109/TTE.2015.2471180
- Sciarretta , A. and Guzzella , L. Optimal Energy-Management Strategies IEEE Control Systems 60 70 April 2007 10.1109/MCS.2007.338280
- Kuznetsova , E. , Li , Y.F. , Ruiz , C. , Zio , E. et al. Reinforcement Learning for Microgrid Energy Management Energy 133 146 Sep. 2013 10.1016/j.energy.2013.05.060
- Liu , T. , Hu , X. , Li , S.E. , and Cao , D. Reinforcement Learning Optimized Look-Ahead Energy Management of a Parallel Hybrid Electric Vehicle IEEE/ASME Transactions on Mechatronics 1497 1507 Aug. 2017 10.1109/TMECH.2017.2707338
- Zou , Y. , Liu , T. , Liu , D. , and Sun , F. Reinforcement Learning-Based Real-Time Energy Management for a Hybrid Tracked Vehicle Applied Energy 372 382 June 2016 10.1016/j.apenergy.2016.03.082
- Abdelhedi , R. , Lahyani , A. , Ammari , A.C. , Sari , A. , and Venet , P. Reinforcement Learning-Based Power Sharing Between Batteries and Supercapacitors in Electric Vehicles Proceedings of the IEEE International Conference on Industrial Technology 2018 2072 2077
- Ermon , S. , Xue , Y. , Gomes , C. , and Selman , B. Learning Policies for Battery Usage Optimization in Electric Vehicles Machine Learning 177 194 July 2013 10.1007/s10994-013-5378-z
- Hu , X. , Li , S. , and Peng , H. A Comparative Study of Equivalent Circuit Models for Li-ion Batteries Journal of Power Sources 359 367 Jan. 2012 10.1016/j.jpowsour.2011.10.013
- Cordoba-Arenas , A. , Onori , S. , Guezennec , Y. , and Rizzoni , G. Capacity and Power Fade Cycle-Life Model for Plug-in Hybrid Electric Vehicle Lithium-ion Battery Cells Containing Blended Spinel and Layered-Oxide Positive Electrodes Journal of Power Sources 473 483 March 2015 10.1016/j.jpowsour.2014.12.047
- 2020 https://www.maxwell.com/products/ultracapacitors/125v-tran-modules
- Sutton , R. and Barto , A. 2018