This content is not included in your SAE MOBILUS subscription, or you are not logged in.
Using Polygot Persistence with NoSQL Databases for Streaming Multimedia, Sensor, and Messaging Services in Autonomous Vehicles
ISSN: 0148-7191, e-ISSN: 2688-3627
To be published on April 14, 2020 by SAE International in United States
This content contains downloadable datasetsAnnotation ability available
The explosion of big data has created challenges for both cloud-based systems and Autonomous Vehicles (AVs) in data collection and management. The same challenges are now being realized in developing databases for integrated sensors, streaming, real-time and on-demand services in AVs. With just one AV expecting to generate over 30 Terabytes of data a day, modern NoSQL databases provide opportunities to horizontally scale AV data seamlessly. NoSQL provides solutions designed to accommodate a wide variety of data models such as, key-value, document, column and graph databases. Key-value stores are by nature scalable, fast processing, and distribute horizontally. These databases are tasked with handling several data types including IoT, radar, lidar, ultra-sonic sensors, GPS, odometry, and sensor data while providing streaming and real-time services. NoSQL can store and utilize structured, semi-structured, and unstructured data necessary for multimedia storage needs. NoSQL databases such as Graph databases support big data necessary for the demands of modern software development. Graph databases can scale AV data by using geospatial and geolocation coordinates as entities for flexible queries and pattern recognition. This paper addresses the development of an autonomous platform to process structured, unstructured, semi-structured, and polymorphic data using NoSQL databases built on a hybrid framework. Using Polygot Persistence for processing multimedia, social media, GPS data, audio, fleet diagnostics, and messaging services will be incorporated into the Platform as a Services (PaaS). Integration of NoSQL’s toolboxes and horizontal scalability through cloud deployments will continue the development of an integrated database that’s scalable for the PaaS application.
CitationBrown, K., "Using Polygot Persistence with NoSQL Databases for Streaming Multimedia, Sensor, and Messaging Services in Autonomous Vehicles," SAE Technical Paper 2020-01-0942, 2020.
Data Sets - Support Documents
|[Unnamed Dataset 1]|
- Ismail, N. , “Graph Databases Lie at the Heart of $7TN Self-Driving Car Opportunity,” https://www.information-age.com/graph-databases-heart-self-driving-car-opportunity-123468309/, September 2017.
- Fosić, I. and Šolić, K. , “Graph Database Approach for Data Storing, Presentation and Manipulation,” HEP Telekomunikacije, 2019, MIPRO 2019/SSE.
- Li, Z. , “NoSQL Databases,” The Geographic Information Science & Technology Body of Knowledge, 2018, doi:10.22224/gistbok/2018.2.4.
- Mongo, D.B. , “What Is NoSQL,” https://www.mongodb.com/nosql-explained.
- Jeon, J., An, M., and Lee, H. , “NoSQL Database Modeling for End-of-Life Vehicle Monitoring System,” Journal of Software, 2015, doi:10.17706/jsw.10.10.1160-1169.
- Stefan, E. , “List of NoSQL Databases,” http://nosql-database.org.
- Rossi, B. , “Graph databases: making meaning from the Internet of Things,” https://www.information-age.com/graph-databasesmaking-meaning-internet-things-123458606/, November 2014.
- Neo4j , “A Graph Platform Reveals and Persists Connections,” https://neo4j.com/product/#overview, 2019.
- Alley, G. , “What is Data Streaming?” https://dzone.com/articles/what-is-data-streaming, November 2018.
- Kepner, J., Arcand, W., Bestor, D., Bergeron, B., Byun, C., Gadepally, V., Hubbell, M., Michaleas, P., Mullen, J., Prout, A., Reuther, A., Rosa, A., and Yee, C. , “Achieving 100,000,000 Database Inserts Per Second using Accumulo and D4M,” in High Performance Extreme Computing Conference (HPEC), 2014, IEEE.
- Kepner, J. and Jananthan, H. , Mathematics of Big Data (Cambridge: MIT Press, 2018). ISBN:978-8026203839-3.
- Samsi, S., Brattain, L., Arcand, W., Bestor, D., Bergeron, B., Byun, C., Gadepally, V., Hubbell, M., Jones, M., Klein, A. et al. , “Benchmarking SciDB Data Import on HPC Systems,” in High Performance Extreme Computing Conference (HPEC), 2016, 1-5, IEEE.
- Sen, R., Farris, A., and Guerra, P. , “Benchmarking Apache Accumulo Big Data Distributed Table Store using Its Continuous Test Suite,” in Big Data (Big Data Congress), 2013 IEEE International Congress on, 2013, 334-341, IEEE.
- CrateDB , “Big Bite: Ingesting Performance of Large Clusters,” https://crate.io/a/big-cluster-insights-ingesting/, April 2016.