Coupled Lateral–Longitudinal Trajectory Tracking Control for Autonomous Vehicles Based on Multi-Agent Reinforcement Learning

Trajectory tracking control serves as the core operational component of autonomous vehicles, directly determining driving safety and passenger comfort by ensuring control precision and stability. To enhance the tracking accuracy and stability for autonomous vehicles, this study proposes a coupled lateral–longitudinal trajectory tracking controller based on multi-agent reinforcement learning. The framework first establishes a Model predictive controller (MPC) derived from vehicle dynamics, formulating the lateral control process as a Markov decision process. A reward function incorporating lateral error, heading error, and steering angle is designed, followed by the construction of a Deep Q-Network (DQN) Agent to optimize the prediction horizon of the MPC. Subsequently, a position–velocity dual-loop PID controller is developed for longitudinal control, with its parameter optimization strategy learned through a Deep Deterministic Policy Gradient (DDPG) Agent. The Extended State Observer (ESO) is incorporated to perform steering angle compensation for internal modeling errors and external disturbances. Co-simulation experiments are conducted in CarSim and MATLAB/Simulink, and the results demonstrate that the coupled controller achieves superior tracking accuracy and stability in both overtaking and lane-changing scenarios compared with the decoupled controller.