This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Toward Unsupervised Test Scenario Extraction for Automated Driving Systems from Urban Naturalistic Road Traffic Data
Journal Article
12-06-03-0017
ISSN: 2574-0741, e-ISSN: 2574-075X
Sector:
Topic:
- Level 3 (Conditional driving automation),
- Level 4 (High driving automation),
- Level 5 (Full driving automation),
- Automated driving systems,
- Machine learning,
- Big data,
- Artificial intelligence (AI),
- Test procedures,
- Vehicle to vehicle (V2V),
- Driver behavior,
- Safety testing and procedures,
- Neural networks
Citation:
Weber, N., Thiem, C., and Konigorski, U., "Toward Unsupervised Test Scenario Extraction for Automated Driving Systems from Urban Naturalistic Road Traffic Data," SAE Intl. J CAV 6(3):263-281, 2023, https://doi.org/10.4271/12-06-03-0017.
Language:
English
Abstract:
Scenario-based testing is a promising approach to solving the challenge of
proving the safe behavior of vehicles equipped with automated driving systems
(ADS). Since an infinite number of concrete scenarios can theoretically occur in
real-world road traffic, the extraction of scenarios relevant in terms of the
safety-related behavior of these systems is a key aspect for their successful
verification and validation. Therefore, a method for extracting multimodal urban
traffic scenarios from naturalistic road traffic data in an unsupervised manner,
minimizing the amount of (potentially biased) prior expert knowledge, is
proposed. Rather than an (elaborate) rule-based assignment by extracting
concrete scenarios into predefined functional scenarios, the presented method
deploys an unsupervised machine learning pipeline. The approach allows for
exploring the unknown nature of the data and their interpretation as test
scenarios that experts could not have anticipated. The method is evaluated for
naturalistic road traffic data at urban intersections from the inD and the
Silicon Valley Intersections datasets. For this purpose, it is analyzed with
which clustering approach (K-means, hierarchical clustering,
and DBSCAN) the scenario extraction method performs best (referring to an
elaborate rule-based implementation). Subsequently, using hierarchical
clustering the results show both a jump in the overall accuracy of around 20%
when moving from 4 to 5 clusters and a saturation effect starting at 41 clusters
with an overall accuracy of 84%. These observations can be a valuable
contribution in the context of the trade-off between the number of functional
scenarios (i.e., clustering accuracy) and testing effort. The possible reasons
for the observed accuracy variations of different clusters, each with a fixed
total number of given clusters, are discussed. The findings encourage the use of
this type of data and unsupervised machine learning approaches as valuable
pillars for the systematic construction of a relevant scenario database with
sufficient coverage for testing ADS.