This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Calibration and Stitching Methods of Around View Monitor System of Articulated Multi-Carriage Road Vehicle for Intelligent Transportation
Technical Paper
2019-01-0873
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
The around view monitor (AVM) system for the long-body road vehicle with multiple articulated carriages usually suffers from the incomplete distortion rectification of fisheye cameras and the irregular image stitching area caused by the change of relative position of the cameras on different carriages while the vehicle is in motion. In response to these problems, a set of calibration and stitching methods of AVM are proposed. First, a radial-distortion-based rectification method is adopted and improved. This method establishes two lost functions and solves the model parameters with the two-step optimization method. Then, AVM system calibration is conducted, and the perspective transformation matrix is calculated. After that, a static basic look-up table is generated based on the distortion rectification model and perspective transformation matrix. Furthermore, to solve the problem of the variable relative position of cameras, the relative angles of the carriages are collected by the angle sensors and then transmitted to the central processor via the CAN bus. And then a dynamic offset look-up table is generated in real time. With the two look-up tables, the images captured by the fisheye cameras are mapped to the top view image. Finally, the global color adjustment and weighted fusion are applied to obtain a seamless around top view image. The AVM algorithms are implemented on an embedded system based on the NXP i. mx series SoC. The field experiments are conducted using a three-carriage vehicle with eight cameras. The results show that the proposed algorithms can output the seamless around top view images in real time with globally balanced brightness and color.
Recommended Content
| Journal Article | Control and Validation of an Autonomous Drag Racing Vehicle |
| Journal Article | Road Profile Estimation for Active Suspension Applications |
| Technical Paper | Linear Model Updating Applied to Rotating Machines |
Authors
Topic
Citation
Feng, X., LI, W., Wei, T., Zhang, Y. et al., "Calibration and Stitching Methods of Around View Monitor System of Articulated Multi-Carriage Road Vehicle for Intelligent Transportation," SAE Technical Paper 2019-01-0873, 2019, https://doi.org/10.4271/2019-01-0873.Data Sets - Support Documents
| Title | Description | Download |
|---|---|---|
| Unnamed Dataset 1 | ||
| Unnamed Dataset 2 |
Also In
References
- Guo , C. , Meguro , J. , Kojima , Y. , and Naito , T. A Multimodal ADAS System for Unmarked Urban Scenarios Based on Road Context Understanding IEEE Transactions on Intelligent Transportation Systems 16 4 1690 1704 2015
- Ranney , T.A. , Garrott , W.R. , and Goodman , M.J. NHTSA Driver Distraction Research: Past, Present, and Future SAE Technical Paper 2001-06-0177 2001
- Kim , Z.W. Robust Lane Detection and Tracking in Challenging Scenarios IEEE Transactions on Intelligent Transportation Systems 9 1 16 26 2008
- Lei , Y. and Wu , J. Study of Applying Zig Bee Technology into Forward Collision Warning System (FCWS) under Low-Speed Circumstance Wireless and Optical Communication Conference (WOCC), 2016 25th 2016 1 4 IEEE
- Su , C-L. , Lee , C-J. , Li , M-S. , and Chen , K-P. 3D AVM System for Automotive Applications Information, Communications and Signal Processing (ICICS), 2015 10th International Conference on 2015 1 5 IEEE
- Tao , B.-N. , Yang , P. , Wu , Y. et al. Analysis of Death Incidents in Road Traffic in China from 2004 to 2013 China Public Security: Academic Edition 2016
- Zhang , Z. A Flexible New Technique for Camera Calibration IEEE Transactions on Pattern Analysis & Machine Intelligence 22 11 1330 1334 2000
- Scaramuzza , D , Martinelli , A , and Siegwart , R A Flexible Technique for Accurate Omnidirectional Camera Calibration and Structure from Motion IEEE International Conference on Computer Vision Systems 2006 45 45
- Ying , X. , Mei , X. , Yang , S. , Wang , G. et al. Radial Distortion Correction from a Single Image of a Planar Calibration Pattern Using Convex Optimization Image Processing (ICIP), 2014 IEEE International Conference On 2014 3440 3443 IEEE
- Wang , Z. , Liang , H. , Wu , X. , Zhao , Y. et al. A Practical Distortion Correcting Method from Fisheye Image to Perspective Projection Image Information and Automation, 2015 IEEE International Conference On 1178 1183 IEEE 2015
- Shah , S. and Aggarwal , J.K. Intrinsic Parameter Calibration Procedure for a (High-Distortion) Fish-Eye Lens Camera with Distortion Model and Accuracy Estimation Pattern Recognition 29 11 1775 1788 1996
- Kanatani , K. Calibration of Ultrawide Fisheye Lens Cameras by Eigenvalue Minimization IEEE Transactions on Pattern Analysis and Machine Intelligence 35 4 813 822 2013
- Hartley , R. and Kang , S.B. Parameter-Free Radial Distortion Correction with Center of Distortion Estimation IEEE Transactions on Pattern Analysis and Machine Intelligence 29 8 1309 1321 2007
- Bi , G. , Zhang , X. , Feng , W. , Zhu , J. et al. Error Analysis of Fisheye Correction Curve Mechatronics and Automation (ICMA), 2015 IEEE International Conference On 2015 1659 1664 IEEE
- Carroll , R. , Agrawal , M. , and Agarwala , A. Optimizing Content-Preserving Projections for Wide-Angle Images ACM Transactions on Graphics (TOG) 28 43 2009
- Marcon , M. , Sarti , A. , and Tubaro , S. Piecewise Distortion Correction for Fisheye Lenses Image Processing (ICIP), 2015 IEEE International Conference on 2015 4057 4061 IEEE
- Di F. , Paolo , S.M. , and Mutto , C.D. Real-Time Image Distortion Correction: Analysis and Evaluation of FPGA-Compatible Algorithms ReConFigurable Computing and FPGAs (ReConFig), 2016 International Conference on 2016 1 6 IEEE
- Shum , H.-Y. and Szeliski , R. Construction of Panoramic Image Mosaics with Global and Local Alignment Panoramic Vision Springer 2001 227 268
- Zhu , Z. , Riseman , E.M. , and Hanson , A.R. Parallel-Perspective Stereo Mosaics Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on 1 2001 345 352 IEEE
- Brown , M. and Lowe , D.G. Automatic Panoramic Image Stitching Using Invariant Features International Journal of Computer Vision 74 1 59 73 2007
- Burt , P.J. and Adelson , E.H. A Multiresolution Spline with Application to Image Mosaics ACM Transactions on Graphics (TOG) 2 4 217 236 1983
- Shum , H.-Y. and Szeliski , R. Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment International Journal of Computer Vision 36 2 101 130 2000
- Gao , G. and Jia , K. A New Image Mosaics Algorithm Based on Feature Points Matching Innovative Computing, Information and Control, 2007. ICICIC’07. Second International Conference on 2007 471 471 IEEE
- Szeliski , R. Image Mosaicing for Tele-Reality Applications Applications of Computer Vision, 1994., Proceedings of the Second IEEE Workshop on 1994 44 53 IEEE
- Kim , D.-H. , Yoon , Y.-I. , and Choi , J.-S. An Efficient Method to Build Panoramic Image Mosaics Pattern Recognition Letters 24 14 2421 2429 2003
- Levy , N. 2016
- Wang , Y. , Li , X. , Zhang , S. et al. 2015
- Wang , W. and Ng , M.K. A Variational Method for Multiple-Image Blending IEEE Transactions on Image Processing 21 4 1809 1822 2012
- Liu , H. , Deng , M. , Xiao , C. , and Xu , X. Image Fusion Algorithm Based on Adaptive Weighted Coefficients Signal Processing (ICSP), 2010 IEEE 10th International Conference on 2010 748 751 IEEE
- Sreehari , S. , Venkatakrishnan , S.V. , Bouman , K.L. , Simmons , J.P. et al. Multi-Resolution Data Fusion for Super-Resolution Electron Microscopy CVPR Workshops 2017 1084 1092
- Omar , Z. , Mitianoudis , N. , and Stathaki , T. Region-Based Image Fusion Using a Combinatory Chebyshev-ICA Method Acoustics, Speech and Signal Processing (ICASSP), 2011 IEEE International Conference on 2011 1213 1216 IEEE
- Jiang , Q. , Jin , X. , Hou , J. , Lee , S.-J. et al. Multi-Sensor Image Fusion Based on Interval Type-2 Fuzzy Sets and Regional Features in Nonsubsampled Shearlet Transform Domain IEEE Sensors Journal 18 6 2494 2505 2018
- Wei , Q. , Dobigeon , N. , and Tourneret , J.-Y. Bayesian Fusion of Multi-Band Images IEEE Journal of Selected Topics in Signal Processing 9 6 1117 1127 2015
- Lee , S. , Lee , S.J. , Park , J. , and Jin , H. Exposure Correction and Image Blending for Planar Panorama Stitching Control, Automation and Systems (ICCAS), 2016 16th International Conference on 2016 128 131
- Andrade , J. and Karam , L.J. Robust Radial Distortion Correction Based on Alternate Optimization 2016 2956 2960
- Rufli , M. , Scaramuzza , D. , and Siegwart , R. Automatic Detection of Checkerboards on Blurred and Distorted Images Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on 2008 3121 3126
- Zhang , Q. and Kamata , S.-I. Fisheye Image Correction Based on Straight-Line Detection and Preservation Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on 2015 1793 1797
- Hedouin , R. , Commowick , O. , Bannier , E. , Scherrer , B. et al. Block-Matching Distortion Correction of Echo-Planar Images with Opposite Phase Encoding Directions IEEE Trans. Med. Imaging 36 5 1106 1115 2017
- Weng , J. , Cohen , P. , and Herniou , M. Camera Calibration with Distortion Models and Accuracy Evaluation IEEE Transactions on Pattern Analysis & Machine Intelligence 10 965 980 1992
- Fitzgibbon , A.W. Simultaneous Linear Estimation of Multiple View Geometry and Lens Distortion Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on 2001 1 I I IEEE
- Reinhard , E. , Adhikhmin , M. , Gooch , B. , and Shirley , P. Color Transfer between Images IEEE Computer Graphics and Applications 21 5 34 41 2001
- Wang , Z. , Bovik , A.C. , Sheikh , H.R. , and Simoncelli , E.P. Image Quality Assessment: From Error Visibility to Structural Similarity IEEE Transactions on Image Processing 13 4 600 612 2004