A Field Operation Test (FOT) and Use case scenario recording capture is very expensive due to the amount of necessary material (vehicles, measurement equipment, headcount, data storage capacity/complexity, trained drivers/professionals) and all-time robust working vehicle setup is not available, mileage is directly proportional to time, along with that it cannot be scaled up due to physical limitations. During the early development phase, ground truth data is not available, and data that can be reused from other projects may not match 100% with current project requirements. All event scenarios/weather conditions cannot be ensured during recording capture.
Carla is an autonomous open-source driving simulator, used for the development, training, and validation of autonomous driving systems, by integrating generative AI, particularly Generative Adversarial Networks (GANs) and Retrieval Augmented Generation (RAG) which are deep learning models, the process of creating synthetic data, and vehicle models becomes more efficient and reliable.
A large language model that takes user input in the form of user prompts can generate scenarios that are used to produce a vast amount of high-quality, distinct, and realistic driving scenarios that closely resemble real-world driving data. Generative AI empowers the user to generate not only dynamic environment conditions such as different weather conditions and lighting conditions but also dynamic elements like the behavior of other vehicles and pedestrians.
Virtual recording generated using AI can be used to train and validate virtual vehicle models, FOT/use case data which is used to indirectly prove real-world performance of functionality of tasks such as object detection, and decision-making algorithms in autonomous vehicles.
Augmenting LLM with Carla involves training generative Models on real-world driving data using Retrieval Augmented Generation which allows the model to generate new, synthetic instances that resemble real-world conditions/scenarios closely.