This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Visual SLAM in Long-Range Autonomous Parking Application Based on Instance-Aware Semantic Segmentation via Multi-Task Network Cascades and Metric Learning Scheme
Technical Paper
2021-01-0077
ISSN: 2641-9637, e-ISSN: 2641-9645
This content contains downloadable datasets
Annotation ability available
Sector:
Event:
SAE WCX Digital Summit
Language:
English
Abstract
Long-range Autonomous Parking is becoming an attractive application in terms of demands. The vehicle is capable of driving autonomously into the appointed parking slot when the driver leaves it at the drop-off spot. In this application, the ability of accurate localization has become a key issue, especially in GPS-denied environments. This paper proposes a method of localization and mapping for Long-range Autonomous Parking, which is achieved by Visual SLAM based on deep learning algorithms. Firstly, we propose an instance segmentation via multi-task network cascades, and even in a complex visual environment, the main roadway instances of interest in the parking lot IPM image can be detected, such as parking corners, speed bumps. Then we combine the information of wheel encoders to build a global semantic map of the parking lot. Vehicles can often rely on semantic map matching to achieve high-precision localization. However, without a good initial position, it is difficult to infer an accurate position by matching the semantic map, such as randomly selecting entrances to enter the parking lot. Therefore, we propose an area feature network based on metric learning to extract features that distinguish different areas and infer the approximate initial position of the vehicle. Specifically, we extract features from the images of the surround-view cameras, use the vehicle position as weak supervision, and finally construct an area feature map. In summary, our proposed method provides accurate vehicle localization and parking lot maps for Long-range Autonomous Parking.
Authors
Topic
Citation
Yan, Y., Hang, Y., Hu, T., Yu, H. et al., "Visual SLAM in Long-Range Autonomous Parking Application Based on Instance-Aware Semantic Segmentation via Multi-Task Network Cascades and Metric Learning Scheme," SAE Int. J. Adv. & Curr. Prac. in Mobility 3(3):1357-1368, 2021, https://doi.org/10.4271/2021-01-0077.Data Sets - Support Documents
| Title | Description | Download |
|---|---|---|
| Unnamed Dataset 1 | ||
| Unnamed Dataset 2 | ||
| Unnamed Dataset 3 | ||
| Unnamed Dataset 4 | ||
| Unnamed Dataset 5 | ||
| Unnamed Dataset 6 | ||
| Unnamed Dataset 7 | ||
| Unnamed Dataset 8 |
Also In
SAE International Journal of Advances and Current Practices in Mobility
Number: V130-99EJ; Published: 2021-06-15
Number: V130-99EJ; Published: 2021-06-15
References
- Arras , K.O. and Tomatis , N. Improving Robustness and Precision in Mobile Robot Localization by Using Laser Range Finding and Monocular Vision 1999 Third European Workshop on Advanced Mobile Robots (Eurobot'99). Proceedings 177 185 IEEE 1999
- Jensfelt , P. , Kragic , D. , Folkesson , J. , and Bjorkman , M. A Framework for Vision Based Bearing only 3D SLAM Proceedings 2006 IEEE International Conference on Robotics and Automation 2006 ICRA 2006 1944 1950 IEEE 2006
- Hess , W. , Kohler , D. , Rapp , H. , and Andor , D. Real-Time Loop Closure in 2D LIDAR SLAM 2016 IEEE International Conference on Robotics and Automation (ICRA) 1271 1278 IEEE 2016
- Labbe , M. and Michaud , F. Online Global Loop Closure Detection for Large-Scale Multi-Session Graph-Based SLAM 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems 2661 2666 IEEE 2014
- Chiang , K.-W. , Huang , Y.-W. , Li , C.-Y. , and Chang , H.-W. An ANN Embedded RTS Smoother for an INS/GPS Integrated Positioning and Orientation System Applied Soft Computing 11 2 2633 2644 2011
- Zhang , G. and Hsu , L.-T. Intelligent GNSS/INS Integrated Navigation System for a Commercial UAV Flight Control System Aerospace Science and Technology 80 368 380 2018
- Cho , S.Y. and Kim , B.D. Adaptive IIR/FIR Fusion Filter and its Application to the INS/GPS Integrated System Automatica 44 8 2040 2047 2008
- Nobori , K. , Ukita , N. , and Hagita , N. A Surround View Image Generation Method with Low Distortion for Vehicle Camera Systems using a Composite Projection 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA) 386 389 IEEE 2017
- Lin , C.-C. and Wang , M.-S. A Vision Based Top-View Transformation Model for a Vehicle Parking Assistant Sensors 12 4 4431 4446 2012
- Davison , A.J. , Reid , I.D. , Molton , N.D. , and Stasse , O. MonoSLAM: Real-Time Single Camera SLAM IEEE Transactions on Pattern Analysis and Machine Intelligence 29 6 1052 1067 2007
- Campos , Carlos , Elvira , R. , Gómez Rodríguez , J.J. et al. ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM arXiv preprint arXiv:2007.11898 2020
- Mur-Artal , R. and Tardós , J.D. Visual-Inertial Monocular SLAM with Map Reuse IEEE Robotics and Automation Letters 2 2 796 803 2017
- Qin , T. , Li , P. , and Shen , S. Vins-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator IEEE Transactions on Robotics 34 4 1004 1020 2018
- Silveira , G. , Malis , E. , and Rives , P. An Efficient Direct Approach to Visual SLAM IEEE transactions on robotics 24 5 969 979 2008
- Engel , J. , Schöps , T. , and Cremers , D. LSD-SLAM: Large-Scale Direct Monocular SLAM European Conference on Computer Vision 834 849 Springer Cham 2014
- Lowry , S. , Sünderhauf , N. , Newman , P. , Leonard , J.J. et al. Visual Place Recognition: A Survey IEEE Transactions on Robotics 32 1 1 19 2015
- Cai , Zhaowei and Vasconcelos Nuno Cascade r-cnn: Delving into High Quality Object Detection Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 6154 6162 2018
- Dai , J. , He , K. , Yi , L. , Ren , S. , and Sun , J. Instance-Sensitive Fully Convolutional Networks European Conference on Computer Vision 534 549 Springer Cham 2016
- Long , Jonathan , Shelhamer Evan , and Darrell Trevor Fully Convolutional Networks for Semantic Segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3431 3440 2015
- Papandreou , G. , L-Ch Chen , K. Murphy , and Yuille A.L. Weakly-and Semi-Supervised Learning of a DCNN for Semantic Image Segmentation 2015
- Chen , L.-C. , Papandreou , G. , Kokkinos , I. , Murphy , K. , and Yuille , A.L. Deeplab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected Crfs IEEE Transactions on Pattern Analysis and Machine Intelligence 40 4 834 848 2017
- Gidaris , S. and Komodakis , N. Object Detection via Aulti-Region and Semantic Segmentation-Aware CNN Model Proceedings of the IEEE International Conference on Computer Vision 1134 1142 2015
- Girshick , R. , Donahue , J. , Darrell , T. , and Malik , J. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 580 587 2014
- Hariharan , B. , Arbeláez , P. , Girshick , R. , and Malik , J. Hypercolumns for Object Segmentation and Fine-Grained Localization Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 447 456 2015
- Dai , J. , He , K. , and Sun , Jian. Instance-Aware Semantic Segmentation via Multi-Task Network Cascades Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 3150 3158 2016
- Caruana , R. Multitask Learning Machine learning 28 1 41 75 1997
- Chen , K. , Pang , J. , Wang , J. , Xiong , Y. , Li , X. , Sun , S. , Feng , W. et al. Hybrid Task Cascade for Instance Segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 4974 4983 2019
- Geng , C. and Jiang , X. Fully Automatic Face Recognition Framework Based on Local and Global Features Machine Vision and Applications 24 3 537 549 2013
- Arandjelovic , R. , Gronat , P. , Torii , A. , Pajdla , T. , and Sivic , J. NetVLAD: CNN Architecture for Weakly Supervised Place Recognition Proceedings of the IEEE conference on computer vision and pattern recognition 5297 5307 2016
- Lopez-Antequera , M. , Gomez-Ojeda , R. , Petkov , N. , and Gonzalez-Jimenez , J. Appearance-Invariant Place Recognition by Discriminatively Training a Convolutional Neural Network Pattern Recognition Letters 92 89 95 2017
- Xin , Z. , Cai , Y. , T. , Lu , Xing , X. , Cai , S. , Zhang , J. , Yang , Y. , and Wang , Y. Localizing Discriminative Visual Landmarks for Place Recognition 2019 International Conference on Robotics and Automation (ICRA) 5979 5985 IEEE 2019
- Noh , H. , Araujo , A. , Sim , J. , Weyand , T. , and Han , B. Large-Scale Image Retrieval with Attentive Deep Local Features Proceedings of the IEEE International Conference on Computer Vision 3456 3465 2017
- Thrun , S. Probabilistic Robotics Communications of the ACM 45 3 52 57 2002
- Yuan , Y. , Kuang , H. , and Schwertfeger , S. Fast Gaussian Process Occupancy Maps 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV) 1502 1507 2018
- He , K. , Gkioxari , G. , Dollár , P. , and Girshick , R. Mask R-CNN Proceedings of the IEEE International Conference on Computer Vision 2961 2969 2017
- Cai , Z. and Vasconcelos , N. Cascade R-CNN: Delving into High Quality Object Detection Proceedings of the IEEE conference on computer vision and pattern recognition 6154 6162 2018
- Chen , K. , Pang , J. , Wang , J. , Xiong , Y. , Li , X. , Sun , S. , Feng , W. et al. Hybrid Task Cascade for Instance Segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 4974 4983 2019
- Dai , J. , Qi , H. , Xiong , Y. , Li , Y. , Zhang , G. , Hu , H. , and Wei , Y. Deformable Convolutional Networks Proceedings of the IEEE International Conference on Computer Vision 764 773 2017
- Liu , S. , Qi , L. , Qin , H. , Shi , J. , and Jia , J. Path Aggregation Network for Instance Segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 8759 8768 2018
- Lin , T.-Y. , Maire , M. , Belongie , S. , Hays , J. , Perona , P. , Ramanan , D. , Dollár , P. , and Lawrence Zitnick , C. Microsoft Coco: Common Objects in Context European conference on computer vision 740 755 Springer Cham 2014
- Szegedy , Christian , Ioffe Sergey , Vanhoucke Vincent , and Alemi Alex Inception-V4, Inception-Resnet and the Impact of Residual Connections on Learning 2016
- Xin , Z. , Cai , Y. , T. , Lu , Xing , X. , Cai , S. , Zhang , J. , Yang , Y. , and Wang , Y. Localizing Discriminative Visual Landmarks for Place Recognition 2019 International Conference on Robotics and Automation (ICRA) 5979 5985 IEEE 2019
- Liu , Y. , Feng , R. , and Zhang , H. Keypoint Matching by Outlier Pruning with Consensus Constraint 2015 IEEE International Conference on Robotics and Automation (ICRA) 5481 5486 IEEE 2015
- Bian , J.W. , Lin , W.-Y. , Matsushita , Y. , Yeung , S.-K. , Nguyen , T.-D. , and Cheng , M.-M. Gms: Grid-Based Motion Statistics for Fast, Ultra-Robust Feature Correspondence Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 4181 4190 2017
- Alvarez , J. and Petersson , L. Decomposeme: Simplifying Convnets for End-to-End Learning 2016
- Schroff , F. , Kalenichenko , D. , and Philbin , J. Facenet: A Unified Embedding for Face Recognition and Clustering Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 815 823 2015
- Bellet , A. , Habrard , A. , and Sebban , M. A Survey on Metric Learning for Feature Vectors and Structured Data arXiv preprint arXiv:1306.6709 2013
- Dupuis , M. , Bahram , M. , Grezlikowski , H. , Richter , A. et al. OpenDRIVE Format Specification, Rev 1.3 OpenDRIVE Document VI2014 106 2010
- Lin , T.-Y. , Dollár , P. , Girshick , R. , He , K. , Hariharan , B. , and Belongie , S. Feature Pyramid Networks for Object Detection Proceedings of the IEEE conference on computer vision and pattern recognition 2117 2125 2017