Adaptive Test Feedback Loop: A Modeling Approach for Checking Side Effects during Test Execution in Advised Explorative Testing

The main objective of testing is to evaluate the functionality, reliability, and operational safety of products. However, this objective makes testing a complex and expensive stage in the development process. From the perspective of an aircraft OEM, test cases are used to verify integration, system, and application levels. Therefore, test cases certify the products against the requirements using the black box testing approach. In doing so, a test plan defines a sequence of test cases whereby it sets up the environment, stimulates the fault, and then observes the system under test (SUT) for each case. Subsequently, the postprocessing of the test execution classifies the test plan as passed or failed. The ongoing digitization and interconnectedness between aircraft systems increase the complexity in functional testing. This trend leads to a high number of test cases and a multitude of reasons why a specific test-case fails. A corresponding error analysis and adaptation of the test plan is a complex and lengthy process that starts repeatedly after executing a manually updated test plan. The goal of the developed approach is to reduce the amount and the duration of the time-consuming trial and error approach. Therefore, the paper proposes the advised “free testing” as an extension of existing functional test processes. The proposal enables the adaptive test feedback loop. It covers the adaption of a current test execution as soon as faulty behavior is detected recursively. The adaption is based on proposals, which are calculated on basis of a knowledge base and selected by a test engineer. For this purpose, the paper proposes a new tool landscape. It describes the required functionality for a test bed. Finally, the paper evaluates the benefits of an adaptive test feedback loop based on an avionics test scenario.