Camera-Radar Data Fusion for Target Detection via Kalman Filter and Bayesian Estimation

Target detection is essential to the advanced driving assistance system (ADAS) and automatic driving. And the data fusion of millimeter wave radar and camera could provide more accurate and complete information of targets and enhance the environmental perception performance. In this paper, a method of vehicle and pedestrian detection based on the data fusion of millimeter wave radar and camera is proposed to improve the target distance estimation accuracy. The first step is the targets data acquisition. A deep learning model called Single Shot MultiBox Detector (SSD) is utilized for targets detection in consecutive video frames captured by camera and further optimized for high real-time performance and accuracy. Secondly, the coordinate system of camera and radar are unified by coordinate transformation matrix. Then, the parallel Kalman filter is used to track the targets detected by radar and camera respectively. Since targets data provided by the camera and radar are different, different Kalman filters are designed to achieve the tracking process. Finally, the targets data are fused based on Bayesian Estimation. At first, several simulation experiments were designed to test and optimize the proposed method, then the real data was used to prove further. Through experiments, it shows that the measurement noise can be considerably reduced by Kalman filter and the fusion algorithm could improve the estimation accuracy.