An Augmented around View Monitor System Fusing Depth and Image Information during the Reversing Process

The around view monitor (AVM) system for vehicles usually suffers from the distortion of surrounding objects caused by incomplete rectification and stitching, which seriously affects the driver's judgment of the surrounding environment during the reversing process. In response to solve this problem, an augmented around view monitor (AAVM) system fusing image and depth information is proposed, which highlights the point clouds of persons or vehicles at the rear of the vehicle. First, an around view image is generated from four fisheye cameras. Then, the calibration of multi TOF cameras is conducted to improve their accuracy of depth estimation and obtain extrinsic camera positions. Next, the 2D-driven object point cloud detection method is proposed to localize and segment object point clouds like vehicles or persons. Finally, the around view image, the object point clouds, and the car model are unified in the same vehicle-centric coordinate system, which will be projected at a suitable angle of view so as to easy to observe. The field experiments are conducted using a passenger car with 4 fisheye cameras and 3 TOF cameras. The results show that the AAVM can help the driver to effectively judge the real environment behind the vehicle during the reversing process.