Mild Hybrid Electric Vehicle Powertrain Control Using Offline–Online Hybrid Deep Reinforcement Learning Strategy

For hybrid electric vehicles (HEVs) to operate efficiently, the amount of power drawn from each power source must be optimally controlled in real time. Recent studies have shown that deep reinforcement learning (DRL) can effectively control the power sources in HEVs. However, model-free DRL relies on a large data set sampled from the system for improved performance, which can be very time consuming and resource intensive. To address this issue, a new DRL strategy is presented in this article where existing vehicle data is exploited to pretrain offline neural networks (NNs) and then the trained NNs are combined with an online DRL algorithm to explore new control policy to further improve the fuel economy. In this manner, it is expected that the controller can quickly learn how to control the system optimally rather than learning an optimal control policy by interacting with the vehicle model from zero initial knowledge. With the offline pretrained NNs embedded in the online DRL framework, the proposed approach not only accelerates the learning process, but also leads to a better fuel economy in most of the simulated cases presented in the article.