This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Federated Learning Enable Training of Perception Model for Autonomous Driving
Technical Paper
2024-01-2873
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
For intelligent vehicles, a robust perception system relies on training datasets with a large variety of scenes. The architecture of federated learning allows for efficient collaborative model iteration while ensuring privacy and security by leveraging data from multiple parties. However, the local data from different participants is often not independent and identically distributed, significantly affecting the training effectiveness of autonomous driving perception models in the context of federated learning. Unlike the well-studied issues of label distribution discrepancies in previous work, we focus on the challenges posed by scene heterogeneity in the context of federated learning for intelligent vehicles and the inadequacy of a single scene for training multi-task perception models. In this paper, we propose a federated learning-based perception model training system. Through visual explanation, we delve into the relationship between model convergence direction and the content of local data scenes. We also investigate the complex relationships between different perception tasks and the diverse scenarios encountered by vehicles. Subsequently, by utilizing significance detection, the system identifies scene distribution characteristics in different client-local datasets while strategically forming alliances among different vehicle clients. The system effectively balances the scene heterogeneity in different client data and mitigates the performance degradation caused by the inadequacy of a single scene to provide sufficient information for training multiple tasks simultaneously. In our experiments, the system not only outperforms the traditional federated averaging but also demonstrates performance improvements compared to other federated aggregation method.
Authors
Topic
Citation
Liu, J., Yin, Z., Nie, L., and Zhao, X., "Federated Learning Enable Training of Perception Model for Autonomous Driving," SAE Technical Paper 2024-01-2873, 2024.Also In
References
- Ha T. , Lee G. , Kim D. and Oh S. Road Graphical Neural Networks for Autonomous Roundabout Driving 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Prague, Czech Republic 2021 162 167 10.1109/IROS51168.2021.9636411
- Li , L. et al. A Review of Applications in Federated Learning Computers & Industrial Engineering doi.org/10.1016/j.cie.2020.106854
- Li T. , Sahu A.K. , Talwalkar A. and Smith V. Federated Learning: Challenges, Methods, and Future Directions IEEE Signal Processing Magazine 37 3 50 60 2020 10.1109/MSP.2020.2975749
- Du , Z. , Wu , C. , Yoshinaga , T. , Yau , K.-L.A. et al. Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues IEEE Open Journal of the Computer Society 1 2020 45 61 10.1109/OJCS.2020.2992630
- Tan , A.Z. , Yu , H. , Cui , L. , and Yang , Q. Towards Personalized Federated Learning IEEE Transactions on Neural Networks and Learning Systems 10.1109/TNNLS.2022.3160699
- McMahan , B. et al. Communication-Efficient Learning of Deep Networks from Decentralized Data Artificial Intelligence and Statistics. PMLR 2017
- Li Xiang , Huang Kaixuan , Yang Wenhao , Wang Shusen , and Zhang Zhihua 2019
- Karimireddy , Sai Praneeth , et al. Scaffold: Stochastic Controlled Averaging for Federated Learning International Conference on Machine Learning. PMLR 2020
- Yu , Fisher , et al. Bdd100k: A Diverse Driving Dataset for Heterogeneous Multitask Learning Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020
- Darzidehkalani , Erfan , Ghasemi-Rad Mohammad , and van Ooijen P.M.A. Federated Learning in Medical Imaging: Part II: Methods, Challenges, and Considerations Journal of the American College of Radiology 19 8 2022 975 982 doi.org/10.1016/j.jacr.2022.03.016
- Liu Y. , Yu J.J.Q. , Kang J. , Niyato D. and Zhang S. Privacy-Preserving Traffic Flow Prediction: A Federated Learning Approach IEEE Internet of Things Journal 7 8 7751 7763 2020 10.1109/JIOT.2020.2991401
- Zhang H. , Bosch J. and Olsson H.H. End-to-End Federated Learning for Autonomous Driving Vehicles 2021 International Joint Conference on Neural Networks (IJCNN) Shenzhen, China 2021 1 8 10.1109/IJCNN52387.2021.9533808
- Tian Y. , Wang J. , Wang Y. , Zhao C. , Yao F. and Wang X. Federated Vehicular Transformers and Their Federations: Privacy-Preserving Computing and Cooperation for Autonomous Driving IEEE Transactions on Intelligent Vehicles 7 3 456 465 2022 10.1109/TIV.2022.3197815
- Zhao , P.-C. et al. CCP-Federated Deep Learning based on User Trust Chain in Social IoV Wireless Networks 29 4 2023 1555 1566 doi.org/10.1007/s11276-021-02870-1
- Yang P. et al. FedDD: Federated Double Distillation in IoV 2022 IEEE 96th Vehicular Technology Conference (VTC2022-Fall) London, United Kingdom 2022 1 5 10.1109/VTC2022-Fall57202.2022.10012798
- Lei , Y. , Wang , S.L. , Su , C. , and Ng , T.F. OES-Fed: A Federated Learning Framework in Vehicular Network based on Noise Data Filtering PeerJ Computer Science 8 2022 e1101 10.7717/peerj-cs.1101
- Wang , G. et al. Joint Resource Management for Mobility Supported Federated Learning in Internet of Vehicles Future Generation Computer Systems 129 2022 199 211 doi.org/10.1016/j.future.2021.11.020
- He , Chaoyang , et al. 2021
- Li , T. et al. Federated Optimization in Heterogeneous Networks Proceedings of Machine Learning and Systems 2 2020 429 450
- Li , Xiaoxiao , et al. 2021
- Girshick , Ross , et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2014
- Girshick , R. Fast r-cnn Proceedings of the IEEE international Conference on Computer Vision. 2015
- Ren , S. et al. Faster r-cnn: Towards Real-Time Object Detection with Region Proposal Networks Advances in Neural Information Processing Systems 28 2015
- Redmon , J. et al. You Only Look Once: Unified, Real-Time Object Detection Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2016
- Redmon , J. and Farhadi , A. YOLO9000: Better, Faster, Stronger Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017
- Redmon , Joseph , and Farhadi Ali 2018
- Bochkovskiy , Alexey , Wang Chien-Yao , and Liao Hong-Yuan Mark 2020
- Chen , Liang-Chieh , et al. 2014
- Chen L.-C. , Papandreou G. , Kokkinos I. , Murphy K. and Yuille A.L. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs IEEE Transactions on Pattern Analysis and Machine Intelligence 40 4 834 848 2018 10.1109/TPAMI.2017.2699184
- Chen , Liang-Chieh , et al. 2017
- Xie , E. et al. SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers Advances in Neural Information Processing Systems 34 2021 12077 12090
- Qian Y. , Dolan J.M. and Yang M. DLT-Net: Joint Detection of Drivable Areas, Lane Lines, and Traffic Objects IEEE Transactions on Intelligent Transportation Systems 21 11 4670 4679 2020 10.1109/TITS.2019.2943777
- Wu , D. , Liao , M.W. , Zhang , W.T. et al. YOLOP: You Only Look Once for Panoptic Driving Perception Mach. Intell. Res. 19 2022 550 562 10.1007/s11633-022-1339-y
- Fantauzzo L. et al. FedDrive: Generalizing Federated Learning to Semantic Segmentation in Autonomous Driving 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Kyoto, Japan 2022 11504 11511 10.1109/IROS47612.2022.9981098
- Wang , Chien-Yao , Bochkovskiy Alexey , and Liao Hong-Yuan Mark Scaled-yolov4: Scaling Cross Stage Partial Network Proceedings of the IEEE/cvf Conference on Computer Vision and Pattern Recognition 2021
- He K. , Zhang X. , Ren S. and Sun J. Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition IEEE Transactions on Pattern Analysis and Machine Intelligence 37 9 1904 1916 2015 10.1109/TPAMI.2015.2389824
- Lin , Tsung-Yi , et al. Feature Pyramid Networks for Object Detection Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2017
- Selvaraju , Ramprasaath R. , et al. Grad-cam: Visual Explanations from Deep Networks Via Gradient-Based Localization Proceedings of the IEEE International Conference on Computer Vision 2017
- Chen , Yunpeng , et al. Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution Proceedings of the IEEE/CVF International Conference on Computer Vision 2019
- Gao , Shang-Hua , et al. Highly Efficient Salient Object Detection with 100k Parameters European Conference on Computer Vision Cham: Springer International Publishing 2020