Accuracy Assessment of Regime Recognition Algorithms Using Confusion Matrix

An accurate characterization of the operational flight regime is a key characteristic of the CBM system. In order to establish regime recognition algorithms as the basis for retirement time of aircraft components, algorithms should be validated through a rigorous verification and validation process using dedicated flight test data with sufficient flight conditions and time. For military aircraft, ADS-79D Appendix B provides detailed guidance for validation of regime recognition algorithms, including their accuracy requirements: "… such as 97% or greater recognition of the actual flight regimes,…", and "… any unrecognized regimes would introduce less than 0.5% under-prediction of fatigue damage fraction based on the design usage spectrum." It has been shown very challenging to meet this 97% accuracy requirement due to limitations with the current regime recognition algorithms. One of the major issues encountered is that multiple regimes are identified by the code for a supposedly single regime time. It is seem intuitive that more than one single regime are recognized during the changing flight conditions particularly for transient maneuvers, when a time sliced regime recognition logic is implemented even with the help of more advanced logic added. As a way to alleviate the stringent measure of "black and white" criteria, the confusion matrix is studied with correlation factors using regime load pattern recognitions to establish a method for better assessment of regime recognition accuracy, and eventually, better assessment of fatigue damage fraction. The study compares the improved regime recognition accuracy using extensive flight survey data.