Battery Electric Vehicle Control Strategy for String Stability Based on Deep Reinforcement Learning in V2V Driving

This works presents a Reinforcement Learning (RL) agent to implement a Cooperative Adaptive Cruise Control (CACC) system that simultaneously enhances energy efficiency and comfort, while also ensuring string stability. CACC systems are a new generation of ACC which systems rely on the communication of the so-called ego-vehicle with other vehicles and infrastructure using V2V and/or V2X connectivity. This enables the availability of robust information about the environment thanks to the exchange of information, rather than their estimation or enabling some redundancy of data. CACC systems have the potential to overcome one typical issue that arises with regular ACC, that is the lack of string stability. String stability is the ability of the ACC of a vehicle to avoid unnecessary fluctuations in speed that can cause traffic jams, dampening these oscillations along the vehicle string rather than amplifying them. In this work, a real-time ACC for a Battery Electric Vehicle, based on a Deep Reinforcement Learning algorithm called Deep Deterministic Policy Gradient (DDPG), has been developed, aiming at maximizing energy savings, and improving comfort, thanks to the exchange of information on distance, speed and acceleration through the exploitation of vehicle-to-vehicle technology (V2V). The aforementioned DDPG algorithm is also designed in order to achieve the string stability. It relies on a multi-objective reward function that is adaptive to different driving cycles. The simulation results show how the agent can obtain energy savings up to 11% comparing the first following vehicle and the Lead on standard cycles and good adaptability to driving cycles different from the training one.