J3321_202603 Verification and Validation of AI/ML-Based Systems in Ground Vehicles

This document intends to focus on AI/ML systems deployed on board and/or off board the vehicles, including the methods and tools used anywhere in the life cycle of automotive system development. This includes, for instance, AI/ML systems used in the identification of parameters, calibration of classical or neural network-based control algorithms, system level modeling and validation, data process automation pipeline such as automated labeling and annotation, data quality checks, etc. Furthermore, with the continued success of Generative AI (GenAI) capabilities, it is becoming a common practice to leverage these capabilities beyond just generating synthetic data. Multiple artifacts such as code, testing scenarios, and benchmarks could be developed using GenAI tools, necessitating the additional verification of these “generative” development steps. The fundamental challenge with generative methods is grounding, especially if data, scenes, and tests are used for the development of AI-enabled products and features.

AI and ML systems have been the subject of several national and international standards and guideline documents. A brief selection of these are given in Section 2. One, ISO PAS 8800:2024, focuses on safety over the entire life cycle of AI development and deployment.

This document is intended to serve the following purpose:

a. The document is an information report with no mandatory requirements.

b. The focus of the document is only on V&V of the development life cycle of AI-based components or systems.

c. The V&V methods are general in nature. The methods discussed include, but are not restricted to, safety requirements or properties.