Calibration and Stitching Methods of Around View Monitor System of Articulated Multi-Carriage Road Vehicle for Intelligent Transportation

The around view monitor (AVM) system for the long-body road vehicle with multiple articulated carriages usually suffers from the incomplete distortion rectification of fisheye cameras and the irregular image stitching area caused by the change of relative position of the cameras on different carriages while the vehicle is in motion. In response to these problems, a set of calibration and stitching methods of AVM are proposed. First, a radial-distortion-based rectification method is adopted and improved. This method establishes two lost functions and solves the model parameters with the two-step optimization method. Then, AVM system calibration is conducted, and the perspective transformation matrix is calculated. After that, a static basic look-up table is generated based on the distortion rectification model and perspective transformation matrix. Furthermore, to solve the problem of the variable relative position of cameras, the relative angles of the carriages are collected by the angle sensors and then transmitted to the central processor via the CAN bus. And then a dynamic offset look-up table is generated in real time. With the two look-up tables, the images captured by the fisheye cameras are mapped to the top view image. Finally, the global color adjustment and weighted fusion are applied to obtain a seamless around top view image. The AVM algorithms are implemented on an embedded system based on the NXP i. mx series SoC. The field experiments are conducted using a three-carriage vehicle with eight cameras. The results show that the proposed algorithms can output the seamless around top view images in real time with globally balanced brightness and color.