This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Efficient On-Premises Data Storage Solution for Autonomous Driving
Technical Paper
2021-26-0042
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
Total autonomy will be 100% accident free by testing more than 8 Billion miles of sensor data*. However, testing the software in real driving situations on the road would be extremely time consuming, expensive, cumbersome. Numerous improvements of the algorithms are necessary to perfect the control software for autonomous driving. To train and test the algorithms we need sensor data. Here the challenge is to store and manage petabytes of sensor data across the global locations with data integrity. In autonomous vehicle to maintain the road safety, multiple sensors are mounted with varying levels of maturity. Multisensory data is the process of combining observations from different sources to provide a robust and complete description of an environment and to overcome the limitation in term of availability.
This paper describes an end to end architecture and design of global data ingest, data processing accessing sensor data effectively, data service and data protection method. It gives a scalable and reliable solution that is capable to handle hundreds of petabytes using hundreds of computer cores. Proposed design helps location-independent data Ingest, faster data availability, On-Demand usage of data effectively and recovery from accidental deletion of data. It helps to store exabytes of data in a tiered storage model in different categorizations. This architecture will reduce it to within 3 to 5 days for further processing of data with data integrity & failover management, Having right strategy will be beneficial in mitigating the risks in sensor data management.
Recommended Content
Ground Vehicle Standard | V2X Sensor-Sharing for Cooperative and Automated Driving |
Technical Paper | Development of a Latency Optimized Communication Device for WAVE and SAE Based V2X-Applications |
Technical Paper | An Integrated Timing Analysis Methodology for Real-Time Systems |
Authors
Topic
Citation
samantaray, R., "Efficient On-Premises Data Storage Solution for Autonomous Driving," SAE Technical Paper 2021-26-0042, 2021, https://doi.org/10.4271/2021-26-0042.Also In
References
- Published by the IEEE Computer Society ■ 1536-1268/07/$25.00 © 2007 IEEE
- The Microsoft Modern Data Warehouse
- Oracle White Paper Release2 [“Data Mining”]
- Alexander , D. , Pierpaoli , C. , Basser , P. , and Gee , J. Spatial Transformations of Diffusion Tensor Magnetic Resonance Images
- Simulation-Assisted Machine Learning
- Deist , T.M. , Patti , A. , Wang , Z. , Krane , D. , Sorenson , T. , and Craft , D.
- Koopman , P. , and Wagner , M. Challenges in Autonomous Vehicle Testing and Validation
- Deist , T.M. , Patti , A. , Wang , Z. , Krane , D. , and Sorenson , T. Simulation-Assisted Machine Learning, David Craft Bioinformatics, btz199
- 978-1 -4799-1848-5/15/$31.00 ©2015 IEEE Secure Sensor Data Management Model in a Sensor - Cloud Integration Environment
- Data Management Challenges of Data Intensive Scientific Work Flows
- Deelman , E. , and Chervenak , A. USC Information Sciences Institute deelman@isi.edu annc@isi.edu
- Hirschfield , G.
- Pelley , S. How to Make Data Protection and High Availability for Analytics Fast and Easy | Contributing Authors: Michael Nixon Engineering Snowflake Technology