This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Using Polygot Persistence with NoSQL Databases for Streaming Multimedia, Sensor, and Messaging Services in Autonomous Vehicles
ISSN: 0148-7191, e-ISSN: 2688-3627
Published April 14, 2020 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
The explosion of big data has created challenges for both cloud-based systems and Autonomous Vehicles (AVs) in data collection and management. The same challenges are now being realized in developing databases for integrated sensors, streaming, real-time and on-demand services in AVs. With just one AV expecting to generate over 30 Terabytes of data a day, modern NoSQL databases provide opportunities to horizontally scale AV data seamlessly. NoSQL provides solutions designed to accommodate a wide variety of data models such as, key-value, document, column and graph databases. Key-value stores are by nature scalable, fast processing, and distribute horizontally. These databases are tasked with handling several data types including IoT, radar, lidar, ultra-sonic sensors, GPS, odometry, and sensor data while providing streaming and real-time services. NoSQL can store and utilize structured, semi-structured, and unstructured data necessary for multimedia storage needs. NoSQL databases such as Graph databases support big data necessary for the demands of modern software development. Graph databases can scale AV data by using geospatial and geolocation coordinates as entities for flexible queries and pattern recognition. This paper addresses the development of an autonomous platform to process structured, unstructured, semi-structured, and polymorphic data using NoSQL databases built on a hybrid framework. Using Polygot Persistence for processing multimedia, social media, GPS data, audio, fleet diagnostics, and messaging services will be incorporated into the Platform as a Services (PaaS). Integration of NoSQL’s toolboxes and horizontal scalability through cloud deployments will continue the development of an integrated database that’s scalable for the PaaS application.
CitationBrown, K., "Using Polygot Persistence with NoSQL Databases for Streaming Multimedia, Sensor, and Messaging Services in Autonomous Vehicles," SAE Technical Paper 2020-01-0942, 2020, https://doi.org/10.4271/2020-01-0942.
Data Sets - Support Documents
|Unnamed Dataset 1|
- Ismail , N. https://www.information-age.com/graph-databases-heart-self-driving-car-opportunity-123468309/ September 2017
- Fosić , I. and Šolić , K. 2019
- Li , Z. 2018 10.22224/gistbok/2018.2.4
- Mongo , D.B. https://www.mongodb.com/nosql-explained
- Jeon , J. , An , M. , and Lee , H. NoSQL Database Modeling for End-of-Life Vehicle Monitoring System Journal of Software 2015 10.17706/jsw.10.10.1160-1169
- Stefan , E. http://nosql-database.org
- Rossi , B. https://www.information-age.com/graph-databasesmaking-meaning-internet-things-123458606/ November 2014
- Neo4j https://neo4j.com/product/#overview 2019
- Alley , G. https://dzone.com/articles/what-is-data-streaming November 2018
- Kepner , J. , Arcand , W. , Bestor , D. , Bergeron , B. , Byun , C. , Gadepally , V. , Hubbell , M. , Michaleas , P. , Mullen , J. , Prout , A. , Reuther , A. , Rosa , A. , and Yee , C. Achieving 100,000,000 Database Inserts Per Second using Accumulo and D4M High Performance Extreme Computing Conference (HPEC) 2014
- Kepner , J. and Jananthan , H. Mathematics of Big Data Cambridge MIT Press 2018 978-8026203839-3
- Samsi , S. , Brattain , L. , Arcand , W. , Bestor , D. , Bergeron , B. , Byun , C. , Gadepally , V. , Hubbell , M. , Jones , M. , Klein , A. et al. Benchmarking SciDB Data Import on HPC Systems High Performance Extreme Computing Conference (HPEC) 2016 1 5
- Sen , R. , Farris , A. , and Guerra , P. Benchmarking Apache Accumulo Big Data Distributed Table Store using Its Continuous Test Suite Big Data (Big Data Congress), 2013 IEEE International Congress on 2013 334 341
- CrateDB https://crate.io/a/big-cluster-insights-ingesting/ April 2016