This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Understanding How Rain Affects Semantic Segmentation Algorithm Performance
ISSN: 0148-7191, e-ISSN: 2688-3627
To be published on April 14, 2020 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
Research interests in autonomous driving have increased significantly in recent years. Several methods are being suggested for performance optimization of autonomous vehicles. However, weather conditions such as rain, snow, and fog may hinder the performance of autonomous algorithms. It is therefore of great importance to study how the performance/efficiency of the underlying scene understanding algorithms vary with such adverse scenarios. Semantic segmentation is one of the most widely used scene-understanding techniques applied to autonomous driving. In this work, we study the performance degradation of several semantic segmentation algorithms caused by rain for off-road driving scenes. Given the limited availability of datasets for real-world off-road driving scenarios that include rain, we utilize two types of synthetic datasets. The first dataset is a pure synthetic rainy dataset which considers the rain droplets on a camera lens, which is suitable for an autonomous vehicle with outside-mounted cameras. This data is generated by the MAVS simulator. In the second dataset, we take good-weather imagery and artificially incorporate rain streaks. By investigating different simulated rain rates, we quantify the performance of such algorithms and witness the severe performance degradation with increasing rain density. We also propose and analyze two methods to obtain the robust performance of segmentation algorithms for both clear and rainy weather.
- Suvash Sharma - Mississippi State University
- Chris Goodin - Mississippi State University
- Matthew Doude - Mississippi State University
- Christopher Hudson - Mississippi State University
- Daniel Carruth - Mississippi State University
- Bo Tang - Mississippi State University
- John Ball - Mississippi State Univ
CitationSharma, S., Goodin, C., Doude, M., Hudson, C. et al., "Understanding How Rain Affects Semantic Segmentation Algorithm Performance," SAE Technical Paper 2020-01-0092, 2020.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
- Chen, B.-K., Gong, C., and Yang, J. , “Importance-Aware Semantic Segmentation for Autonomous Driving System,” in IJCAI, 2017.
- De Brabandere, B., Neven, D., and Van Gool, L. , “Semantic Instance Segmentation for Autonomous Driving,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2017.
- Siam, M. et al. , “A Comparative Study of Real-Time Semantic Segmentation for Autonomous Driving,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 2018.
- Sharma, S. et al. , “Semantic Segmentation with Transfer Learning for Off-Road Autonomous Driving,” 19(11):2577, 2019.
- Young, T. et al. , “Recent Trends in Deep Learning Based Natural Language Processing,” 13(3):55-75, 2018.
- Litjens, G. et al. , “A Survey on Deep Learning in Medical Image Analysis,” 42:60-88, 2017.
- Rafi, R.H.M., Tang, B., and Sharma, S. , “Multi-Layer Embedding Neural Architecture with External Memory for Large-Scale Text Categorization,” in 2018 IEEE International Conference on Big Data (Big Data), 2018, IEEE.
- Zhang, Z., Fidler, S., and Urtasun, R. , “Instance-Level Segmentation for Autonomous Driving with Deep Densely Connected Mrfs,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.
- Pohlen, T. et al. , “Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017.
- Drews, P. et al. , “Aggressive Deep Driving: Model Predictive Control with a Cnn Cost Model,” 2017.
- Garcia-Garcia, A. et al. , “A Review on Deep Learning Techniques Applied to Semantic Segmentation,” 2017.
- Luo, Y., Xu, Y., and Ji, H. , “Removing Rain from a Single Image Via Discriminative Sparse Coding,” in Proceedings of the IEEE International Conference on Computer Vision, 2015.
- Yinka, A.O. et al. , “Performance of Drivable Path Detection System of Autonomous Robots in Rain and Snow Scenario,” in 2014 International Conference on Signal Processing and Integrated Networks (SPIN), 2014. IEEE.
- Wang, Y. et al. , “Pointseg: Real-Time Semantic Segmentation Based on 3d Lidar Point Cloud,” 2018.
- Goodin, C. et al. , “Training of Neural Networks with Automated Labeling of Simulated Sensor Data,” SAE Technical Paper 2019-01-0120 , 2019, https://doi.org/10.4271/2019-01-0120.
- Valada, A. et al. “Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion,” in International Symposium on Experimental Robotics, 2016, Springer.
- Brostow, G.J., Fauqueur, J., and Cipolla, R. , “Semantic Object Classes in Video: A High-Definition Ground Truth Database,” 30(2):88-97, 2009.
- Garg, K. and Nayar, S.K. , “Vision and Rain,” 75(1):3-27, 2007.
- Barnum, P.C., Narasimhan, S., and Kanade, T.J. , “Analysis of Rain and Snow in Frequency Space,” 86(2-3):256, 2010.
- Romera, E. et al. , “Bridging the Day and Night Domain Gap for Semantic Segmentation,” in 2019 IEEE Intelligent Vehicles Symposium (IV), 2019, IEEE.
- Sun, L. et al. , “See Clearer at Night: Towards Robust Nighttime Semantic Segmentation through Day-Night Image Conversion,” in Artificial Intelligence and Machine Learning in Defense Applications, 2019, International Society for Optics and Photonics.
- Badrinarayanan, V. et al. , “Segnet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation,” 39(12):2481-2495, 2017.
- Long, J., Shelhamer, E., and Darrell, T. , “Fully Convolutional Networks for Semantic Segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015.
- Valada, A. et al. , “Adapnet: Adaptive Semantic Segmentation in Adverse Environmental Conditions,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), 2017, IEEE.
- He, K. et al. , “Deep Residual Learning for Image Recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016.