Recognition Assistance Interface for Human-Automation Cooperation in Pedestrian Risk Prediction

Autonomous driving systems (ADS) have been widely tested in real-world environments with operators who must monitor and intervene due to remaining technical challenges. However, intervention methods that require operators to take over control of the vehicle involve many drawbacks related to human performance. ADS consist of recognition, decision, and control modules. The latter two phases are dependent on the recognition phase, which still struggles with tasks involving the prediction of human behavior, such as pedestrian risk prediction. As an alternative to full automation of the recognition task, cooperative recognition approaches utilize the human operator to assist the automated system in performing challenging recognition tasks, using a recognition assistance interface to realize human-machine cooperation. In this study, we propose a recognition assistance interface for cooperative recognition in order to achieve safer and more efficient driving through improved human-automation cooperation. A simulator experiment with 18 participants is conducted to evaluate our recognition assistance interface in comparison with a conventional control intervention, in terms of driving safety, efficiency, and usability. Recognition of pedestrian crossing intention is selected for the cooperation task, and driving scenarios in which the automated system cannot reliably recognize the crossing intentions of pedestrians at non-signalized locations are selected as the driving scenario. Statistical analysis of our experimental results reveals that the proposed recognition assistance interface allowed more accurate operator intervention, was easier to use, and achieved more stable vehicle control than the control intervention. We also found that sharing the recognition information of the automated driving system with operators could divide their attention, impairing intervention performance. Our experimental results suggest that the unifying presentation of the system recognition information and the operator’s manipulation target on the touchscreen of the user interface addresses this problem.