This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
An Augmented around View Monitor System Fusing Depth and Image Information during the Reversing Process
Technical Paper
2020-01-0095
ISSN: 0148-7191, e-ISSN: 2688-3627
This content contains downloadable datasets
Annotation ability available
Sector:
Language:
English
Abstract
The around view monitor (AVM) system for vehicles usually suffers from the distortion of surrounding objects caused by incomplete rectification and stitching, which seriously affects the driver's judgment of the surrounding environment during the reversing process. In response to solve this problem, an augmented around view monitor (AAVM) system fusing image and depth information is proposed, which highlights the point clouds of persons or vehicles at the rear of the vehicle. First, an around view image is generated from four fisheye cameras. Then, the calibration of multi TOF cameras is conducted to improve their accuracy of depth estimation and obtain extrinsic camera positions. Next, the 2D-driven object point cloud detection method is proposed to localize and segment object point clouds like vehicles or persons. Finally, the around view image, the object point clouds, and the car model are unified in the same vehicle-centric coordinate system, which will be projected at a suitable angle of view so as to easy to observe. The field experiments are conducted using a passenger car with 4 fisheye cameras and 3 TOF cameras. The results show that the AAVM can help the driver to effectively judge the real environment behind the vehicle during the reversing process.
Authors
Citation
LI, W., Wang, W., Jiang, L., Xiao, D. et al., "An Augmented around View Monitor System Fusing Depth and Image Information during the Reversing Process," SAE Technical Paper 2020-01-0095, 2020, https://doi.org/10.4271/2020-01-0095.Data Sets - Support Documents
| Title | Description | Download |
|---|---|---|
| Unnamed Dataset 1 | ||
| Unnamed Dataset 2 | ||
| Unnamed Dataset 3 |
Also In
References
- Guo , C. , Meguro , J. , Kojima , Y. , and Naito , T. A Multimodal ADAS System for Unmarked Urban Scenarios Based on Road Context Understanding IEEE Transactions on Intelligent Transportation Systems 16 4 1690 1704 2015
- Kuo , T. , Wei , Y. , Cheng , Y. , and Cheng , F. 3D around View System with Ground Estimation 2018 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW) IEEE 2018 1 2
- Kim , Z. Robust Lane Detection and Tracking in Challenging Scenarios IEEE Transactions on Intelligent Transportation Systems 9 1 16 26 2008
- Dagan , E. , Mano , O. , Stein , G.P. , and Shashua , A. Forward Collision Warning with a Single Camera 2004 IEEE Intelligent Vehicles Symposium IEEE 2004 37 42
- Wu , B.-F. , Kao , C.-C. , Li , Y.-F. , and Tsai , M.-Y. A Real-Time Embedded Blind Spot Safety Assistance System International Journal of Vehicular Technology 506235 2012
- Greenhalgh , J. and Mirmehdi , M. Real-Time Detection and Recognition of Road Traffic Signs IEEE Transactions on Intelligent Transportation Systems 13 4 1498 1506 2012
- Chen , Y.-Y. , Tu , Y.-Y. , Chiu , C.-H. , and Chen , Y.-S. An Embedded System for Vehicle Surrounding Monitoring 2009 2nd International Conference on Power Electronics and Intelligent Transportation System (PEITS) IEEE 2009 92 95
- Zhang , Z. A Flexible New Technique for Camera Calibration IEEE Transactions on Pattern Analysis and Machine Intelligence 22 11 1330 1334 2000
- Chen , H. , Yu , H. , and Long , A. A New Camera Calibration Algorithm Based on Rotating Object Robot Vision: Second International Workshop, 2008. Rob Vis 2008 403 411
- Ying , X. , Mei , X. , Yang , S. , Wang , G. et al. Radial Distortion Correction from a Single Image of a Planar Calibration Pattern Using Convex Optimization 2014 IEEE International Conference on Image Processing (ICIP) IEEE 2014 3440 3443
- Marcon , M. , Sarti , A. , and Tubaro , S. Piecewise Distortion Correction for Fisheye Lenses 2015 IEEE International Conference on Image Processing (ICIP) IEEE 2015 4057 4061
- Shum , H.-Y. and Szeliski , R. Construction and Refinement of Panoramic Mosaics with Global and Local Alignment Sixth International Conference on Computer Vision (ICCV) IEEE 1998 953 956
- Brown , M. and Lowe , D. Automatic Panoramic Image Stitching Using Invariant Features International Journal of Computer Vision 74 59 73 2007
- Wang , W. and Ng , M. A Variational Method for Multiple-Image Blending IEEE Transactions on Image Processing 21 1809 1822 2011
- Liu , H. , Deng , M. , Xiao , C. , and Xu , X. Image Fusion Algorithm Based on Adaptive Weighted Coefficients 2010 IEEE 10th International Conference on Signal Processing Proceedings IEEE 2010 748 751
- Kuo , T.-Y. , Wang , Y.-S. , Cheng , Y.-J. , and Wan , K.-H. 3D around View Monitoring System for Automobiles 2017 IEEE International Conference on Consumer Electronics (ICCE) IEEE 2017 271 272
- Suhr , J.K. and Jung , H.G. Sensor Fusion-Based Vacant Parking Slot Detection and Tracking IEEE +Transactions on Intelligent Transportation Systems 15 1 21 36 2014
- Park , M.W. , Jang , K.H. , and Jung , S.K. Panoramic Vision System to Eliminate Driver’s Blind Spots Using a Laser Sensor and Cameras International Journal of Intelligent Transportation Systems Research 10 3 101 114 2012
- Feng , X. , Li , W. , Wei , T. , Zhang , Y. et al. Calibration and Stitching Methods of around View Monitor System of Articulated Multi-Carriage Road Vehicle for Intelligent Transportation SAE Technical Paper 2019-01-0873 2019 https://doi.org/10.4271/2019-01-0873
- Scaramuzza , D. , Martinelli , A. , and Siegwart , R. A Flexible Technique for Accurate Omnidirectional Camera Calibration and Structure from Motion Fourth IEEE International Conference on Computer Vision Systems (ICVS'06) IEEE 2006 45 45
- Shanghai Data Miracle Intelligent Tech. Ltd. https://github.com/smarttofsdk/SDK
- Heikkilä , J. Silvén , O. A Four-Step Camera Calibration Procedure with Implicit Image Correction 1997 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR) IEEE 1997 1106 1112
- Jean-Yves , B. http://www.vision.caltech.edu/bouguetj/calib_doc/
- Schiller , I. , Beder , C. , and Koch , R. Calibration of a PMD-Camera Using a Planar Calibration Pattern Together with a Multi-Camera Setup Proceedings of the XXI ISPRS Congress 2008
- Redmon , J. and Farhadi , A. 2018
- Lahoud , J. and Ghanem , B. 2D-driven 3D Object Detection in RGB-D Images 2017 IEEE International Conference on Computer Vision (ICCV) IEEE 2017 4632 4640
- Ester , M. , Kriegel , H.-P. , Sander , J. , and Xu , X. A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise Proceedings of 2nd International Conference on Knowledge Discovery and Data Mining (KDD-96) AAAI Press 1996 226 231
- Zhou , Q.-Y. , Park , J. , and Koltun , V. 2018