This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Optimization of Energy Management Strategy for Range-Extended Electric Vehicle Using Reinforcement Learning and Neural Network
Technical Paper
2020-01-1190
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
A Range-Extended Electric Vehicle (REEV) uses battery as the primary energy source and engine as the secondary source to extend the total range of the vehicle. Deep Orange 11 program at Clemson University is proposing a REEV for solving the mobility needs in the year 2040. Designing the Energy Management System (EMS) of such a vehicle is a critical aspect of the problem statement of this program to improve the vehicle economy and bring down the cost of operation of the vehicle. This paper proposes a reinforcement learning based algorithm for designing the EMS of such a vehicle. Q-learning is a model-free algorithm which seeks to improve the cumulative reward by finding the best policy over the course of operation. A rule-based strategy is first used to establish a baseline model of engine operation during the operation of vehicle over an EPA drive-cycle (FHDS). The Q-learning strategy is then deployed which learns over the baseline strategy as the vehicle travels over the drive cycle and improves the fuel economy of the vehicle. A high-fidelity vehicle simulator is developed in Simulink to simultaneously train and validate the strategy. Finally, a neural network is used to approximate the relationship between state and actions which can be used for implementation of the strategy in the vehicle, owing to its higher computational efficiency. The target of the research is to cover 120 km most efficiently over FHDS drive cycle. The results of this study showed an improved vehicle economy of 27.78% over the conventional rule-based strategy. The reward function has been designed such that it can be tuned to improve the economy over different driving distances as per requirement. The study also presents a state-based randomization as opposed to a conventional time-based randomization which shows significant improvement in convergence to optimal value.
Recommended Content
Topic
Citation
Mittal, N., Pundlikrao Bhagat, A., Bhide, S., Acharya, B. et al., "Optimization of Energy Management Strategy for Range-Extended Electric Vehicle Using Reinforcement Learning and Neural Network," SAE Technical Paper 2020-01-1190, 2020, https://doi.org/10.4271/2020-01-1190.Data Sets - Support Documents
| Title | Description | Download |
|---|---|---|
| Unnamed Dataset 1 | ||
| Unnamed Dataset 2 |
Also In
References
- Van Batavia , B. Hydraulic Hybrid Vehicle Energy Management System SAE Technical Paper 2009-01-1772 2009 https://doi.org/10.4271/2009-01-1772
- Folkesson , A. , Neuman , M. , and Gravesen , C. Targets, Constraints and Rules for Hybrid Management in a Series Hybrid Bus Intended for Commercial Introduction SAE Technical Paper 2008-01-1563 2008 https://doi.org/10.4271/2008-01-1563
- Jeong , J. , Lee , W. , Kim , N. , Stutenberg , K. et al. Control Analysis and Model Validation for BMW i3 Range Extender SAE Technical Paper 2017-01-1152 2017 https://doi.org/10.4271/2017-01-1152
- Jalil , N. , Kheir , N.A. , and Salman , M. A Rule-Based Energy Management Strategy for a Series Hybrid Vehicle Proceedings of the 1997 American Control Conference 1997 1 689 693
- Trovão , J.P. , Pereirinha , P.G. , Jorge , H.M. , and Antunes , C.H. A Multi-Level Energy Management System for Multisource Electric Vehicles: An Integrated Rule-Based Meta-Heuristic Approach Appl Energy 105 304 318 2013
- Bellman , R. Dynamic Programming Courier Corporation 2013
- Sciarretta , A. , Back , M. , and Guzzella , L. Optimal Control of Parallel Hybrid Electric Vehicles IEEE Transactions on Control Systems Technology 12 352 363 2004
- Onori , S. , Serrao , L. , and Rizzoni , G. Pontryagin’s Minimum Principle Hybrid Electric Vehicles Springer 2016 51 63
- Feng , L. , Cheng , M. , and Chen , B. Predictive Control of a Power-Split HEV with Fuel Consumption and SOC Estimation SAE Technical Paper 2015-01-1161 2015 https://doi.org/10.4271/2015-01-1161
- Lee , H. , Cha , S. , Kim , H. , and Kim , S. Energy Management Strategy of Hybrid Electric Vehicle Using Stochastic Dynamic Programming SAE Technical Paper 2015-01-0019 2015 https://doi.org/10.4271/2015-01-0019
- Nidubrolu , K. , Dhamija , L. , Chaudhary , P. , and Kirtane , C. An Optimal Rule Based Energy Management System for a Hybrid Electric 2-Wheeler Based on Dynamic Programming Technique SAE Technical Paper 2019-26-0128 2019 https://doi.org/10.4271/2019-26-0128
- Chandrasekar , A. , Sengupta , S. , Hingane , S. , Gururaja , C. et al. Comparative Analysis of Model Predictive Control (MPC) and Conventional Control in Supervisory Controller of a Retrofit HEV SAE Technical Paper 2017-26-0093 2017 https://doi.org/10.4271/2017-26-0093
- Xiong , R. , Cao , J. , and Yu , Q. Reinforcement Learning- Based Real-Rime Power Management for Hybrid Energy Storage System in the Plug-In Hybrid Electric Vehicle Applied Energy 211 538 548 2018
- Fang , Y. , Song , C. , Xia , B. , and Song , Q. An Energy Management Strategy for Hybrid Electric Bus Based on Reinforcement Learning Control and Decision Conference (CCDC), 2015 27th Chinese 2015 4973 4977
- Liu , T. , Zou , Y. , Liu , D. , and Sun , F. Reinforcement Learning of Adaptive Energy Management with Transition Probability for a Hybrid Electric Tracked Vehicle IEEE Transactions on Industrial Electronics 62 12 7837 7846 Dec. 2015
- Wu , J. , Zou , Y. , Liu , T. , Kong , Z. et al. An Online Correction Predictive EMS for a Hybrid Electric Tracked Vehicle Based on Dynamic Programming and Reinforcement Learning 10.1109/ACCESS.2019.2926203
- n.d. http://cuicardeeporange.com/project/do11/
- Wang , Y. , Jiao , X. , Sun , Z. , and Li , P. Energy Management Strategy in Consideration of Battery Health for PHEV via Stochastic Control and Particle Swarm Optimization Algorithm Energies 10 1894 2017 10.3390/en10111894
- n.d. https://www3.epa.gov/otaq/gvg/learn-more-technology.htm
- Xu , B. , Malmir , F. , Rathod , D. , and Filipi , Z. Real-Time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle SAE Technical Paper 2019-01-1208 2019 https://doi.org/10.4271/2019-01-1208
- Xu , B. , Rathod , D. , Zhang , D. , Yebi , A. et al. Parametric Study on Reinforcement Learning Optimized Energy Management Strategy for a Hybrid Electric Vehicle Applied Energy 114200 2019