Interaction-Aware Deep Reinforcement Learning Approach Based on Hybrid Parameterized Action Space for Autonomous Driving

Learning-based motion planning methods such as reinforcement learning (RL) have shown great potential of improving the performance of autonomous driving. However, comprehensively ensuring safety and efficiency remain a challenge for motion planning technology. Most current RL methods output discrete behavioral action or continuous control action, which lack an intuitive representation of the future motion and then face the problems with unstable or reckless driving behavior. To address these issues, this work proposes an interaction-aware reinforcement learning approach based on hybrid parameterized action space for autonomous driving in lane change scenario. The proposed method can output high-level feasible trajectory and low-level actuator control command to control the vehicle’s motion together. Meanwhile, the reward functions for the local traffic environment are designed to evaluate the effect of the interaction between ego vehicle and surrounding vehicles. The contributions of the proposed method are: 1) propose a hybrid parameterized action based interaction-aware DRL framework (HPA-IDRL); 2) the proposed HPA-IDRL can learn from the reward not only considering self-benefits but also considering the benefits of the local traffic environment; 3) A multi-head attention layer is embedded before actor network and critic network respectively to exploit the interactive information in the traffic environment. Thus, the HPA-IDRL agent can generate more flexible and smooth driving behavior, which improves the safety and the efficiency of autonomous driving. The proposed method is implemented and validated with other four advanced DRL model in various simulation environments. The results demonstrate that the proposed HPA-IDRL can effectively balance the flexibility and smoothness of driving behavior, leading to the improving performance that is both safe and efficient.