LLM-Powered Fuzz Testing of Automotive Diagnostic Protocols

Modern vehicles contain tens of different Electronic Control Units (ECUs) from several vendors. These small computers are connected through several networking busses and protocols, potentially through gateways and converters. Moreover, vehicle-to-vehicle and internet connectivity are now considered to be requirements, adding an addi- tional layer of complexity to an already complex electronic system. Due to this complexity and the safety-critical nature of vehicles, au- tomotive cyber-security a difficult undertaking. One critical aspect of cyber-security is the robust testing of software for potential bugs and vulnerabilities. Fuzz testing is an automated software testing method that injects a large set of inputs to a system. It is an invaluable tech- nique across many industries, and has been increasingly gaining popu- larity since its conception. Its success highly relies on the ”quality” of inputs injected. One shortcoming associated with fuzz testing is the expertise required in developing ”smart” fuzz testing tools (fuzzers). Developing a fuzzer requires knowledge on a range of topics, from input types and underly- ing networks to potential system configurations. Moreover, fuzzers are generally not transferable between different systems. This study inves- tigates whether Generative AI technologies can meaningfully assist in this development. An automotive fuzzer with a particular focus on Unified Diagnostic Services (UDS) was developed by exclusively querying an AI model. First, the pre-trained AI is taught the underlying structure and con- straints of UDS, and is then used to generate semantically valid test cases. The effectiveness of the test cases is evaluated for vulnerability and fault detection. The impact of specific queries and underlying net- work configurations on the generated test cases are then investigated through comparison with a state-of-the-art fuzzer.