Synthesizing Data for Autonomous Driving: Multi-Agent Reinforcement Learning Meets Augmented Reality

Synthetic data holds significant potential for improving the efficiency of perception tasks in autonomous driving. This paper proposes a practical data synthesis pipeline that employs multi-agent reinforcement learning (MARL) to automatically generate dynamic traffic participant trajectories and leverages augmented reality (AR) processes to produce photo-realistic images. This AR process blends clean static background images extracted from real photos using image matting techniques, with dynamic foreground images rendered from 3D Computer Aided Design (CAD) models in a rendering engine. We posit that this data synthetic pipe line has strong image photorealism, flexible way of interaction scenarios generation and mature tool chain, which has the prospect of large-scale engineering application.