This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Robust Sensor Fused Object Detection Using Convolutional Neural Networks for Autonomous Vehicles
ISSN: 0148-7191, e-ISSN: 2688-3627
Published April 14, 2020 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
Environmental perception is considered an essential module for autonomous driving and Advanced Driver Assistance System (ADAS). Recently, deep Convolutional Neural Networks (CNNs) have become the State-of-the-Art with many different architectures in various object detection problems. However, performances of existing CNNs have been dropping when detecting small objects at a large distance. To deploy any environmental perception system in real world applications, it is important that the system achieves high accuracy regardless of the size of the object, distance, and weather conditions. In this paper, a robust sensor fused object detection system is proposed by utilizing the advantages of both vision and automotive radar sensors. The proposed system consists of three major components: 1) the Coordinate Conversion module, 2) Multi level-Sensor Fusion Detection (MSFD) system, and 3) Temporal Correlation filtering module. The proposed MSFD system employs the principles of artificial intelligence beyond simple comparison of data variance of the sensors. And then, its performance is further improved by using the temporal correlation information with an adaptive threshold scheme. The proposed system is evaluated with the collected video data (6,854 image frames with 18,918 labeled objects). Based on the laboratory testing and in-vehicle validation, the proposed system demonstrates its high accuracy for detecting any size of objects in real-world data.
CitationPark, J., Jayachandran Raguraman, S., Aslam, A., and Gotadki, S., "Robust Sensor Fused Object Detection Using Convolutional Neural Networks for Autonomous Vehicles," SAE Technical Paper 2020-01-0100, 2020, https://doi.org/10.4271/2020-01-0100.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
|[Unnamed Dataset 2]|
- Bi, X., Tan, B., Xu, Z., and Huang, L. , “A New Method of Target Detection Based on Autonomous Radar and Camera Data Fusion,” SAE Technical Paper 2017-01-1977, 2017, https://doi.org/10.4271/2017-01-1977.
- Wang, T., Zheng, N., Xin, J., and Ma, Z. , “Integrating Millimeter Wave Radar with a Monocular Vision Sensor for On-Road Obstacle Detection Applications,” Sensors, 2011.
- Kim, D.Y. and Jeon, M. , “Data Fusion of Radar and Image Measurements for Multi-Object Tracking via Kalman Filtering,” Information Sciences 278:641-652, doi:10.1016/j.ins.2014.03.080.
- Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. , “You Only Look Once:Unified, Real-Time Object Detection,” in IEEE Conference on CVPR, 2016, 779-788.
- Ren, S., He, K., Girshick, R., and Sun, J. , “Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks,” IEEE Transactions on Pattern Analysis and Machine Intelligence 39(6):1137-1149, 1 June 2017, doi:10.1109/TPAMI.2016.2577031.
- Liu, W., Anguelov, D., Erhan, D., Szegedy, C. et al. , “SSD: Single Shot Multi Box Detector,” 2016, arXiv:1512.02325.
- Szegedy, C., Liu, W., Jia, Y., Sermanet, P. et al. , “Going Deeper With Convolutions,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 1-9, 10.1109, 2015.
- Autonomou Stuff , “Delphi ESR Startup Guide,” https://www.autonomoustuff.com/wp-content/uploads/2016/08/delphi_esr_data_sheet.pdf.
- Howard, A., Zhu, M., Chen, B., Kalenichenko, D. et al. , “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” 2017, ar Xiv: 1704.04861.