Novel Diagnostic Metrics for Regime Recognition Verification and Validation

Lifecycle management for many commercial and military helicopters involves the process of regime recognition (RR), in which the maneuver history of the aircraft is classified into individual regimes over the duration of a flight. When considering verification and validation (V&V) for regime recognition codes, a key step lies in assessing the RR code's conformance to defined classification and damage prediction accuracy requirements. The confusion matrix is typically used for this purpose; however, the confusion matrix does not quantify certain critical aspects of RR code performance and oftentimes fails to provide actionable information about how a code may be improved. This paper describes a novel set of V&V diagnostic metrics designed to highlight areas of deficiencies in the algorithm and specific paths to code improvement. These metrics leverage pattern recognition techniques to align the flown and recognized maneuvers and are proposed to supplement and enhance the V&V information obtained from the confusion matrix. Several example V&V flight sequences are provided to highlight the benefits of the proposed diagnostic metrics when used in concert with other classical V&V tools.