Driver Visual Focus of Attention Estimation in Autonomous Vehicles

An existing challenge in current state-of-the-art autonomous vehicles is the process of safely transferring control from autonomous driving mode to manual mode because the driver may be distracted with secondary tasks. Such distractions may impair a driver’s situational awareness of the driving environment which will lead to fatal outcomes during a handover. Current state-of-the-art vehicles notify a user of an imminent handover via auditory, visual, and physical alerts but are unable to improve a driver’s situational awareness before a handover is executed. The overall goal of our research team is to address the challenge of providing a driver with relevant information to regain situational awareness of the driving task. In this paper, we introduce a novel approach to estimating a driver’s visual focus of attention using a 2D RGB camera as input to a Multi-Input Convolutional Neural Network with shared weights. The system was validated in a realistic driving scenario. The developed approach is a first step towards estimating a driver’s situational awareness from their observable indicators which will in the future be utilized to provide targeted information to the driver during a handover.