Multi-Objective Adaptive Cruise Control via Deep Reinforcement Learning

This work presents a multi-objective adaptive cruise control (ACC) system via deep reinforcement learning (DRL). During the control period, it quantitatively considers three indexes: tracking accuracy, riding comfort, and fuel economy. The system balances contradictions between different indexes to achieve the best overall control results. First, a hierarchical control architecture is utilized, where the upper level controller is synthesized under DRL framework to give out the vehicle desired acceleration. The lower level controller executes the command and compensates vehicle dynamics. Then, four state variables that can comprehensively determine the car-following states are selected for better convergence. Multi-objective reward function is quantitatively designed referring to the evaluation indexes, in which safety constraints are considered by adding violation penalty. Thereafter, the training environment which excludes the disturbance of preceding car acceleration is built. And the upper level controller is trained in randomly initialized conditions. Finally, the developed ACC system is tested under typical car-following scenarios and medium speed driving cycles. Simulation results show that the developed ACC system has better overall control performance than the traditional cascade PID method.