This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Optimization of Energy Management Strategy for Range-Extended Electric Vehicle Using Reinforcement Learning and Neural Network
ISSN: 0148-7191, e-ISSN: 2688-3627
Published April 14, 2020 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
A Range-Extended Electric Vehicle (REEV) uses battery as the primary energy source and engine as the secondary source to extend the total range of the vehicle. Deep Orange 11 program at Clemson University is proposing a REEV for solving the mobility needs in the year 2040. Designing the Energy Management System (EMS) of such a vehicle is a critical aspect of the problem statement of this program to improve the vehicle economy and bring down the cost of operation of the vehicle. This paper proposes a reinforcement learning based algorithm for designing the EMS of such a vehicle. Q-learning is a model-free algorithm which seeks to improve the cumulative reward by finding the best policy over the course of operation. A rule-based strategy is first used to establish a baseline model of engine operation during the operation of vehicle over an EPA drive-cycle (FHDS). The Q-learning strategy is then deployed which learns over the baseline strategy as the vehicle travels over the drive cycle and improves the fuel economy of the vehicle. A high-fidelity vehicle simulator is developed in Simulink to simultaneously train and validate the strategy. Finally, a neural network is used to approximate the relationship between state and actions which can be used for implementation of the strategy in the vehicle, owing to its higher computational efficiency. The target of the research is to cover 120 km most efficiently over FHDS drive cycle. The results of this study showed an improved vehicle economy of 27.78% over the conventional rule-based strategy. The reward function has been designed such that it can be tuned to improve the economy over different driving distances as per requirement. The study also presents a state-based randomization as opposed to a conventional time-based randomization which shows significant improvement in convergence to optimal value.
CitationMittal, N., Pundlikrao Bhagat, A., Bhide, S., Acharya, B. et al., "Optimization of Energy Management Strategy for Range-Extended Electric Vehicle Using Reinforcement Learning and Neural Network," SAE Technical Paper 2020-01-1190, 2020, https://doi.org/10.4271/2020-01-1190.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
|[Unnamed Dataset 2]|
- Van Batavia, B. , “Hydraulic Hybrid Vehicle Energy Management System,” SAE Technical Paper 2009-01-1772, 2009, doi:https://doi.org/10.4271/2009-01-1772.
- Folkesson, A., Neuman, M., and Gravesen, C. , “Targets, Constraints and Rules for Hybrid Management in a Series Hybrid Bus Intended for Commercial Introduction,” SAE Technical Paper 2008-01-1563, 2008, doi:https://doi.org/10.4271/2008-01-1563.
- Jeong, J., Lee, W., Kim, N., Stutenberg, K. et al. , “Control Analysis and Model Validation for BMW i3 Range Extender,” SAE Technical Paper 2017-01-1152, 2017, doi:https://doi.org/10.4271/2017-01-1152.
- Jalil, N., Kheir, N.A., and Salman, M. , “A Rule-Based Energy Management Strategy for a Series Hybrid Vehicle,” in: Proceedings of the 1997 American Control Conference (Cat. No. 97CH36041), 1997, 1, 689±693.
- Trovão, J.P., Pereirinha, P.G., Jorge, H.M., and Antunes, C.H. , “A Multi-Level Energy Management System for Multisource Electric Vehicles: An Integrated Rule-Based Meta-Heuristic Approach,” Appl Energy 105:304-318, 2013.
- Bellman, R. , Dynamic Programming (Courier Corporation, 2013).
- Sciarretta, A., Back, M., and Guzzella, L. , “Optimal Control of Parallel Hybrid Electric Vehicles,” IEEE Transactions on Control Systems Technology 12:352-363, 2004.
- Onori, S., Serrao, L., and Rizzoni, G. , “Pontryagin’s Minimum Principle,” in: Hybrid Electric Vehicles (Springer, 2016), 51-63.
- Feng, L., Cheng, M., and Chen, B. , “Predictive Control of a Power-Split HEV with Fuel Consumption and SOC Estimation,” SAE Technical Paper 2015-01-1161, 2015, doi:https://doi.org/10.4271/2015-01-1161.
- Lee, H., Cha, S., Kim, H., and Kim, S. , “Energy Management Strategy of Hybrid Electric Vehicle Using Stochastic Dynamic Programming,” SAE Technical Paper 2015-01-0019, 2015, doi:https://doi.org/10.4271/2015-01-0019.
- Nidubrolu, K., Dhamija, L., Chaudhary, P., and Kirtane, C. , “An Optimal Rule Based Energy Management System for a Hybrid Electric 2-Wheeler Based on Dynamic Programming Technique,” SAE Technical Paper 2019-26-0128, 2019, doi:https://doi.org/10.4271/2019-26-0128.
- Chandrasekar, A., Sengupta, S., Hingane, S., Gururaja, C. et al. , “Comparative Analysis of Model Predictive Control (MPC) and Conventional Control in Supervisory Controller of a Retrofit HEV,” SAE Technical Paper 2017-26-0093, 2017, doi:https://doi.org/10.4271/2017-26-0093.
- Xiong, R., Cao, J., and Yu, Q. , “Reinforcement Learning- Based Real-Rime Power Management for Hybrid Energy Storage System in the Plug-In Hybrid Electric Vehicle,” Applied Energy 211:538-548, 2018.
- Fang, Y., Song, C., Xia, B., and Song, Q. , “An Energy Management Strategy for Hybrid Electric Bus Based on Reinforcement Learning,” in Control and Decision Conference (CCDC), 2015 27th Chinese, 2015, 4973-4977.
- Liu, T., Zou, Y., Liu, D., and Sun, F. , “Reinforcement Learning of Adaptive Energy Management with Transition Probability for a Hybrid Electric Tracked Vehicle,” IEEE Transactions on Industrial Electronics 62(12):7837-7846, Dec. 2015.
- Wu, J., Zou, Y., Liu, T., Kong, Z. et al. , “An Online Correction Predictive EMS for a Hybrid Electric Tracked Vehicle Based on Dynamic Programming and Reinforcement Learning,” IEEE, doi:10.1109/ACCESS.2019.2926203.
- Deep Orange 11 - CU-ICAR Deep Orange, n.d., Retrieved from http://cuicardeeporange.com/project/do11/.
- Wang, Y., Jiao, X., Sun, Z., and Li, P. , “Energy Management Strategy in Consideration of Battery Health for PHEV via Stochastic Control and Particle Swarm Optimization Algorithm,” Energies 10:1894, 2017, doi:10.3390/en10111894.
- Technology | Green Vehicle Guide, n.d., Retrieved from https://www3.epa.gov/otaq/gvg/learn-more-technology.htm.
- Xu, B., Malmir, F., Rathod, D., and Filipi, Z. , “Real-Time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle,” SAE Technical Paper 2019-01-1208, 2019, doi:https://doi.org/10.4271/2019-01-1208.
- Xu, B., Rathod, D., Zhang, D., Yebi, A. et al. , “Parametric Study on Reinforcement Learning Optimized Energy Management Strategy for a Hybrid Electric Vehicle,” Applied Energy 114200, 2019.