On Board Evaluation System for Advanced Driver Assistance Systems

The evaluation of Advanced Driver Assistance Systems (ADAS including driver assistance and active safety) has increasing interest from authorities, industry and academia. AsPeCSS active safety project concludes that good results in a laboratory test for active safety system design does not necessarily equate to an effective system in real traffic conditions. Moreover, many ADAS assessment projects and standards require physical testing on test tracks (dummy vehicles, pedestrian mannequins…), which are expensive and limit testing capabilities. This research presents a conceptual framework for on-board evaluation (OBE) of ADAS, which can be used as a cost effective evaluation in real-life traffic conditions. OBE shall monitor, record, analyze and report both internal behavior and external environment (external objects list and video stream) of ADAS under evaluation (ADASUE). Furthermore, OBE performs white box testing and black box testing (evaluates overall vehicle’s actions during hazards and evaluates functionalities and performance of ADASUE according to predefined test scenarios (e.g. ISO 22178 active safety test protocols)). In order to test that ADASUE is able to detect environmental hazards within adequate response time, a supplementary sub-system of the OBE called hazard detection unit (HDU) is proposed. HDU uses some active safety systems to verify that hazards are detected by ADASUE. Some well-known machine learning (ML) algorithms are used in HDU implementation. The trigger to start evaluation using OBE is based on warnings from HDU, ADASUE itself and connected vehicles. Experiments are performed using an ADAS simulation tool. Finally, results are presented to demonstrate the potential of proposed system.