This article presents a new machine learning (ML) development lifecycle which
will constitute the core of the new aeronautical standard on ML called AS6983,
jointly being developed by working group WG-114/G34 of European Organisation for
Civil Aviation Equipment (EUROCAE) and SAE. The article also presents a survey
of several existing standards and guidelines related to ML in aeronautics,
automotive, and industrial domains by comparing and contrasting their scope,
purpose, and results. Standards and guidelines reviewed include the European
Union Aviation Safety Agency (EASA) Concept Paper, the DEEL
(DEpendable and Explainable Learning)
white paper “Machine Learning in Certified Systems”, Aerospace Vehicle System
Institute (AVSI) Authorization for Expenditure (AFE) 87 report on Machine
Learning, Guidance on the Assurance of Machine Learning for use in Autonomous
Systems (AMLAS), Laboratoire National de Metrologie et d’Essais (LNE)
Certification Standard of Processes for AI, the Underwriters Laboratories (UL)
4600 Safety Standard for Autonomous Vehicles, and the paper on Assuring the
Machine Learning Lifecycle. These standards and guidelines are examined from the
perspective of the learning assurance objectives they propose, and the means of
evaluation and compliance for achieving these learning objectives. The reference
used for comparison is the list of learning assurance objectives defined within
the framework of AS6983 development. From this comparative analysis, and based
on a coverage criterion defined in this article, only three (3) standards and
guidelines exceed 50% coverage of the Machine Learning Development Lifecycle
(MLDL) learning assurance objectives baseline. The next steps of this work are
to update the AS6983 learning assurance objectives and improve the associated
means of compliance to approach a coverage score of 100%, and offer a
certification-based process to other domains that could benefit from the AS6983
standard.
