Development of a Soft-Actor Critic Reinforcement Learning Algorithm for the Energy Management of a Hybrid Electric Vehicle

In recent years, the urgent need to fully exploit the fuel economy potential of the Electrified Vehicles (xEVs) through the optimal design of their Energy Management System (EMS) have led to an increasing interest in Machine Learning (ML) techniques. Among them, Reinforcement Learning (RL) seems to be one of the most promising approaches thanks to its peculiar structure, in which an agent is able to learn the optimal control strategy through the feedback received by a direct interaction with the environment. Therefore, in this study, a new Soft Actor-Critic agent (SAC), which exploits a stochastic policy, was implemented on a digital twin of a state-of-the-art diesel Plug-in Hybrid Electric Vehicle (PHEV) available on the European market. The SAC agent was trained to enhance the fuel economy of the PHEV while guaranteeing its battery charge sustainability. The potential of the proposed control strategy was firstly assessed on the Worldwide harmonized Light-duty vehicles Test Cycle (WLTC) through a benchmark against a Dynamic Programming (DP) optimization and comparing the performance of two different rewards. Then, the best-performing agent was tested on two additional driving cycles taken from the Environmental Protection Agency (EPA) regulatory framework: the Federal Test Procedure-75 (FTP75) and the Highway Fuel Economy Test (HFET), representative of urban and highway driving scenarios, respectively. The best-performing SAC model achieved results close to the DP reference, with a limited gap (lower than 9%) in terms of fuel consumption over all the testing cycles.