Safety Assurance for Automated Driving Systems: Open Problems and Learnings from a Review of Other Domains

Automated Driving Systems (ADSs) for road vehicles will be capable of performing the entire Dynamic Driving Task (DDT) without the active involvement of a human driver. Further, many ADSs will use Machine Learning (ML) to progressively adapt their driving functionality during in-service operation. This presents challenges for traditional regulatory frameworks, which do not readily support automated driving without a human driver or support safety-critical systems using ML to modify driving functionality post-market entry. However, these challenges are not entirely unique to ADSs. A review was undertaken into approaches taken in other domains to assure safety-critical systems that enable automated operation and adaptive functionality. Other transport modes were reviewed, including adaptive flight control systems in aviation, autonomous ship control systems in maritime, and automated train operation in rail. Non-transport domains were also reviewed, including medical devices in healthcare, control systems in nuclear power plants (NPPs), and autonomous weapons in the military. A range of findings are presented, covering issues relating to domain-specific functional safety standards, whole-of-life assurance, autonomous operation, and adaptive system changes. The lack of specific requirements or guidance relating to whether ML-enabled safety-critical changes could occur autonomously online within predetermined boundaries or should be restricted to only occur offline with human oversight was a specific issue identified as deserving of further research. A key recommendation is that the development of ADS safety assurance frameworks should give due consideration to learnings and open problems identified in other domains, while being cognizant that the differences and unique aspects of road transport mean not all findings will readily translate.