This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Reinforcement Learning based Energy Management of Multi-Mode Plug-in Hybrid Electric Vehicles for Commuter Route
Technical Paper
2020-01-1189
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
Optimization-based (OB) methods used in vehicle energy management strategies (EMSs) have the potential to significantly increase fuel economy and extend the electric-only range of plug-in hybrid electric vehicles (PHEVs). However, OB methods are difficult to apply to current real-world vehicles because accurate detailed and high-resolution information about the future, including second-by-second vehicle velocity trajectory data, are not currently available in the current transportation infrastructure. In this paper, a practical reinforcement learning (RL) algorithm for automatic mode-switching of a multimode PHEV is introduced. The PHEV used in the work was a 2016 Chevrolet Volt driven on a simulated commuter route. The goal is to blend the charge depleting and charge sustaining modes during the trip to reduce gasoline consumption and extend electric-only range. The RL algorithm was first trained offline on recorded trips and then used in real-time when the vehicle was driven on a new trip of the same route. While OB methods like dynamic programming can find globally optimal solutions given complete information about a future trip, the RL method developed here does not require detailed future trip information and still obtains substantial improvements. Results show that the fuel economy on a miles per gallon equivalent (MPGe) basis was improved between 5.5% and 6.4% for a tested commuter route as a function of starting battery state of charge using the developed algorithm. The developed method provides an immediate solution to extend electric-only range in PHEVs used on daily commuter routes.
Recommended Content
Authors
Citation
Wang, P. and Northrop, W., "Reinforcement Learning based Energy Management of Multi-Mode Plug-in Hybrid Electric Vehicles for Commuter Route," SAE Technical Paper 2020-01-1189, 2020, https://doi.org/10.4271/2020-01-1189.Data Sets - Support Documents
Title | Description | Download |
---|---|---|
Unnamed Dataset 1 |
Also In
References
- Martinez , C.M. , Hu , X. , Cao , D. et al. Energy Management in Plug-In Hybrid Electric Vehicles: Recent Progress and a Connected Vehicles Perspective IEEE Transactions on Vehicular Technology 66 6 4534 4549 2016
- Wirasingha , S.G. and Emadi , A. Classification and Review of Control Strategies for Plug-In Hybrid Electric Vehicles IEEE Transactions on Vehicular Technology 60 1 111 122 2010
- Chen , Z. , Mi , C.C. , Xu , J. , Gong , X. et al. Energy Management for a Power-Split Plug-In Hybrid Electric Vehicle Based on Dynamic Programming and Neural Networks IEEE Transactions on Vehicular Technology 63 4 1567 1580 2013
- Anselma , P.G. , Huo , Y. , Roeleveld , J. , Belingardi , G. et al. Integration of On-Line Control in Optimal Design of Multimode Power-Split Hybrid Electric Vehicle Powertrains IEEE Transactions on Vehicular Technology 68 4 3436 3445 2019
- Stroe , N. , Olaru , S. , Colin , G. , Ben-Cherif , K. et al. Predictive Control Framework for HEV: Energy Management and Free-Wheeling Analysis IEEE Transactions on Intelligent Vehicles 4 2 220 231 2019
- Tianheng , F. , Yang , L. , Qing , G. , Yanqing , H. et al. A Supervisory Control Strategy for Plug-In Hybrid Electric Vehicles Based on Energy Demand Prediction and Route Preview IEEE Transactions on Vehicular Technology 64 5 1691 1700 2014
- Wang , H. , Huang , Y. , Soltani , A. , Khajepour , A. , and Cao , D. Cyber-Physical Predictive Energy Management for Through-the-Road Hybrid Vehicles IEEE Transactions on Vehicular Technology 68 4 3246 3256 2019
- Li , L. , Coskun , S. , Zhang , F. , Langari , R. et al. Energy Management of Hybrid Electric Vehicle Using Vehicle Lateral Dynamic in Velocity Prediction IEEE Transactions on Vehicular Technology 68 4 3279 3293 2019
- Rama , N. , Wang , H. , Orlando , J. , Robinette , D. et al. Route-Optimized Energy Management of Connected and Automated Multi-Mode Plug-In Hybrid Electric Vehicle Using Dynamic Programming SAE Technical Paper 2019-01-1209 2019 https://doi.org/10.4271/2019-01-1209
- Zhang , S. , Luo , Y. , Wang , J. , Wang , X. et al. Predictive Energy Management Strategy for Fully Electric Vehicles Based on Preceding Vehicle Movement IEEE Transactions on Intelligent Transportation Systems 18 11 3049 3060 2017
- Wu , G. , Boriboonsomsin , K. , and Barth , M.J. Development and Evaluation of an Intelligent Energy-Management Strategy for Plug-In Hybrid Electric Vehicles IEEE Transactions on Intelligent Transportation Systems 15 3 1091 1100 2014
- Sun , C. , Moura , S.J. , Hu , X. , Hedrick , J.K. et al. Dynamic Traffic Feedback Data Enabled Energy Management in Plug-In Hybrid Electric Vehicles IEEE Transactions on Control Systems Technology 23 3 1075 1086 2014
- Zhang , F. , Xi , J. , and Langari , R. Real-Time Energy Management Strategy Based on Velocity Forecasts Using V2V and V2I Communications IEEE Transactions on Intelligent Transportation Systems 18 2 416 430 2016
- Xu , B. , Malmir , F. , Rathod , D. , and Filipi , Z. Real-Time Reinforcement Learning Optimized Energy Management for a 48V Mild Hybrid Electric Vehicle SAE Technical Paper 2019-01-1208 2019 https://doi.org/10.4271/2019-01-1208
- Lin , X. , Bogdan , P. , Chang , N. , and Pedram , M. Machine Learning-Based Energy Management in a Hybrid Electric Vehicle to Minimize Total Operating Cost 2015 IEEE/ACM International Conference on Computer-Aided Design (ICCAD) IEEE 2015 627 634
- Liu , C. and Murphey , Y.L. Power Management for Plug-in Hybrid Electric Vehicles Using Reinforcement Learning with Trip Information 2014 IEEE Transportation Electrification Conference and Expo (ITEC) IEEE 2014 1 6
- Liu , T. , Zou , Y. , Liu , D. , and Sun , F. Reinforcement Learning of Adaptive Energy Management with Transition Probability for a Hybrid Electric Tracked Vehicle IEEE Transactions on Industrial Electronics 62 12 7837 7846 2015
- Xu , B. , Rathod , D. , Zhang , D. , Yebi , A. et al. Parametric Study on Reinforcement Learning Optimized Energy Management Strategy for a Hybrid Electric Vehicle Applied Energy 114200 2019
- Zhao , P. , Wang , Y. , Chang , N. , Zhu , Q. , and Lin , X. A Deep Reinforcement Learning Framework for Optimizing Fuel Economy of Hybrid Electric Vehicles 2018 23rd Asia and South Pacific Design Automation Conference (ASP-DAC) IEEE 2018 196 202
- Liu , T. , Hu , X. , Li , S.E. , and Cao , D. Reinforcement Learning Optimized Look-Ahead Energy Management of a Parallel Hybrid Electric Vehicle IEEE/ASME Transactions on Mechatronics 22 4 1497 1507 2017
- Wu , J. , He , H. , Peng , J. , Li , Y. , and Li , Z. Continuous Reinforcement Learning of Energy Management with Deep Q Network for a Power Split Hybrid Electric Bus Applied energy 222 799 811 2018
- Han , X. , He , H. , Wu , J. , Peng , J. , and Li , Y. Energy Management Based on Reinforcement Learning with Double Deep Q-Learning for a Hybrid Electric Tracked Vehicle Applied Energy 254 113708 2019
- Li , Y. , He , H. , Peng , J. , and Wang , H. Deep Reinforcement Learning-Based Energy Management for a Series Hybrid Electric Vehicle Enabled by History Cumulative Trip Information IEEE Transactions on Vehicular Technology 68 8 7416 7430 2019
- Hu , Y. , Li , W. , Xu , K. , Zahid , T. et al. Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning Applied Sciences 8 2 187 2018
- Zou , Y. , Liu , T. , Liu , D. , and Sun , F. Reinforcement Learning-Based Real-Time Energy Management for a Hybrid Tracked Vehicle Applied Energy 171 372 382 2016
- Xiong , R. , Cao , J. , and Quanqing , Y. Reinforcement Learning-Based Real-Time Power Management for Hybrid Energy Storage System in the Plug-In Hybrid Electric Vehicle Applied Energy 211 538 548 2018
- Qi , X. , Wu , G. , Boriboonsomsin , K. , and Barth , M.J. A Novel Blended Real-Time Energy Management Strategy for Plug-In Hybrid Electric Vehicle Commute Trips 2015 IEEE 18th International Conference on Intelligent Transportation Systems IEEE 2015 1002 1007
- Chen , Z. , Liang , L. , Hu , X. , Yan , B. , and Yang , C. Temporal-Difference Learning-Based Stochastic Energy Management for Plug-in Hybrid Electric Buses IEEE Transactions on Intelligent Transportation Systems 20 6 2378 2388 2018
- Wang , P. , Li , Y. , Shekhar , S. , and Northrop , W.F. A Deep Reinforcement Learning Framework for Energy Management of Extended Range Electric Delivery Vehicles 2019 IEEE Intelligent Vehicles Symposium (IV) IEEE 2019 1837 1842
- Wang , P. , Li , Y. , Shekhar , S. , and Northrop , W.F. Actor-Critic based Deep Reinforcement Learning Framework for Energy Management of Extended Range Electric Delivery Vehicles 2019 IEEE/ASME Advanced Intelligent Mechatronics (AIM) IEEE/ASME 2019 1379 1384
- Hu , X. , Liu , T. , Qi , X. , and Barth , M. Reinforcement Learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management: Recent Advances and Prospects IEEE Industrial Electronics Magazine 13 3 16 25 2019
- Brooker , A. , Gonder , J. , Wang , L. , Wood , E. et al. FASTSim: A Model to Estimate Vehicle Efficiency, Cost, and Performance SAE Technical Paper 2015-01-0973 2015 https://doi.org/10.4271/2015-01-0973
- O'Keefe , M.P. , Simpson , A. , Kelly , K.J. , and Pedersen , D.S. Duty Cycle Characterization and Evaluation towards Heavy Hybrid Vehicle Applications SAE Technical Paper 2007-01-0302 https://doi.org/10.4271/2007-01-0302
- Sutton , R.S. and Barto , A.G. Reinforcement Learning: An Introduction MIT Press 2018
- Mnih , V. , Kavukcuoglu , K. , Silver , D. , Rusu , A.A. et al. Human-Level Control through Deep Reinforcement Learning Nature 518 7540 529 2015
- Van Hasselt , H. , Guez , A. , and Silver , D. Deep Reinforcement Learning with Double q-Learning Thirtieth AAAI Conference on Artificial Intelligence 2016
- Bellman , R. Dynamic Programming Science 153 3731 34 37 1966
- Nair , V. and Hinton , G.E. Rectified Linear Units Improve Restricted Boltzmann Machines Proceedings of the 27th International Conference on Machine Learning (ICML-10) 2010 807 814
- Kingma , D.P. and Ba , J. arXiv preprint arXiv:1412.6980 2014